2023-10-13T23:46:50.474 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-13T23:46:50.475 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_dis@teuthology --name dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi --archive /home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259 --description rbd/cli/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} data-pool/none features/journaling msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd_support_module_recovery} -- /home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259/orig.config.yaml 2023-10-13T23:46:50.506 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-13T23:46:50.589 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259 branch: wip-dis-testing description: rbd/cli/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} data-pool/none features/journaling msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd_support_module_recovery} email: idryomov@gmail.com first_in_suite: false job_id: '7427259' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-dis-testing ceph: conf: client: rbd default features: 125 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 debug rbd: 20 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 01fff6a72a328459c1d153e5dc1de6a34e48a82f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: osd default pool size: 2 install: ceph: flavor: default sha1: 01fff6a72a328459c1d153e5dc1de6a34e48a82f thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-dis-testing-qa sha1: 735f68d7bf91d643813a8420fc2b11441f4bd8c4 owner: scheduled_dis@teuthology priority: 74 repo: https://github.com/ceph/ceph-ci roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2449 sha1: 01fff6a72a328459c1d153e5dc1de6a34e48a82f sleep_before_teardown: 0 subset: 49/99 suite: rbd suite_branch: wip-dis-testing-qa suite_path: /home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa suite_relpath: qa suite_repo: https://github.com/idryomov/ceph suite_sha1: 735f68d7bf91d643813a8420fc2b11441f4bd8c4 targets: smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH/vh7O2KCB/0ptwMSzzgY+G3dboygsXf4i9tzvtMR/JZ7YzaIu2HcgDfLzHJ7Tqa+552PeXue3WWel93DCIE20= tasks: - install: null - ceph: null - install: extra_system_packages: - fio - workunit: clients: client.0: - rbd/rbd_support_module_recovery.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-13_23:35:38 tube: smithi user: dis verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-13T23:46:50.589 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa; will attempt to use it 2023-10-13T23:46:50.590 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa/tasks 2023-10-13T23:46:50.590 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-13T23:46:50.592 INFO:teuthology.task.internal:Checking packages... 2023-10-13T23:46:50.618 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '01fff6a72a328459c1d153e5dc1de6a34e48a82f' 2023-10-13T23:46:50.618 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-13T23:46:50.619 INFO:teuthology.packaging:ref: None 2023-10-13T23:46:50.619 INFO:teuthology.packaging:tag: None 2023-10-13T23:46:50.619 INFO:teuthology.packaging:branch: wip-dis-testing 2023-10-13T23:46:50.620 INFO:teuthology.packaging:sha1: 01fff6a72a328459c1d153e5dc1de6a34e48a82f 2023-10-13T23:46:50.620 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-dis-testing 2023-10-13T23:46:50.772 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:46:50.773 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-13T23:46:50.779 INFO:teuthology.task.internal:no buildpackages task found 2023-10-13T23:46:50.779 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-13T23:46:50.820 INFO:teuthology.task.internal:Saving configuration 2023-10-13T23:46:50.831 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-13T23:46:50.844 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-13T23:46:50.871 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-13 23:41:15.894935', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAT+PQN9Wm2XnzveQWb7q4Y80Wh6InYTx3CxxVLq0ud0'} 2023-10-13T23:46:50.871 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-13T23:46:50.877 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2023-10-13T23:46:50.877 INFO:teuthology.run_tasks:Running task console_log... 2023-10-13T23:46:51.019 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2d3495b940>, signals=[15]) 2023-10-13T23:46:51.021 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-13T23:46:51.026 INFO:teuthology.task.internal:Opening connections... 2023-10-13T23:46:51.026 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2023-10-13T23:46:51.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T23:46:51.107 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-13T23:46:51.114 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2023-10-13T23:46:51.211 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2023-10-13T23:46:51.211 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2023-10-13T23:46:51.258 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-13T23:46:51.259 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="Ubuntu" 2023-10-13T23:46:51.259 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="22.04" 2023-10-13T23:46:51.259 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-13T23:46:51.260 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_CODENAME=jammy 2023-10-13T23:46:51.260 INFO:teuthology.orchestra.run.smithi183.stdout:ID=ubuntu 2023-10-13T23:46:51.260 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE=debian 2023-10-13T23:46:51.260 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-13T23:46:51.260 INFO:teuthology.orchestra.run.smithi183.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-13T23:46:51.261 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-13T23:46:51.261 INFO:teuthology.orchestra.run.smithi183.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-13T23:46:51.261 INFO:teuthology.orchestra.run.smithi183.stdout:UBUNTU_CODENAME=jammy 2023-10-13T23:46:51.262 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2023-10-13T23:46:51.283 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-13T23:46:51.292 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-13T23:46:51.312 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-13T23:46:51.312 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2023-10-13T23:46:51.317 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-13T23:46:51.336 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-13T23:46:51.337 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2023-10-13T23:46:51.368 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-13T23:46:51.400 INFO:teuthology.run_tasks:Running task kernel... 2023-10-13T23:46:51.414 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-13T23:46:51.415 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-13T23:46:51.415 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-13T23:46:51.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-13T23:46:51.420 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2023-10-13T23:46:51.466 INFO:teuthology.orchestra.run.smithi183.stdout:5.15.0-56-generic 2023-10-13T23:46:51.467 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.15.0-56-generic 2023-10-13T23:46:51.467 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2023-10-13T23:46:51.605 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2023-10-13T23:46:51.742 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-10-13T23:46:51.858 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-13T23:46:51.940 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-13T23:46:51.975 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [856 kB] 2023-10-13T23:46:52.070 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [346 kB] 2023-10-13T23:46:52.076 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [175 kB] 2023-10-13T23:46:52.078 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-10-13T23:46:52.079 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [953 kB] 2023-10-13T23:46:52.099 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.0 kB] 2023-10-13T23:46:52.100 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [154 kB] 2023-10-13T23:46:52.102 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [532 B] 2023-10-13T23:46:52.102 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [789 kB] 2023-10-13T23:46:52.112 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [560 kB] 2023-10-13T23:46:52.117 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [145 kB] 2023-10-13T23:46:52.118 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.7 kB] 2023-10-13T23:46:52.143 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-10-13T23:46:52.144 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-10-13T23:46:52.144 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-10-13T23:46:52.144 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-10-13T23:46:52.293 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-10-13T23:46:52.433 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,062 kB] 2023-10-13T23:46:52.701 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [506 kB] 2023-10-13T23:46:52.741 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [234 kB] 2023-10-13T23:46:52.756 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.0 kB] 2023-10-13T23:46:52.758 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.3 kB] 2023-10-13T23:46:52.760 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [974 kB] 2023-10-13T23:46:52.811 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [157 kB] 2023-10-13T23:46:52.818 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [532 B] 2023-10-13T23:46:52.981 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [991 kB] 2023-10-13T23:46:53.599 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [658 kB] 2023-10-13T23:46:53.637 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [217 kB] 2023-10-13T23:46:53.650 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.0 kB] 2023-10-13T23:46:53.651 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-10-13T23:46:53.654 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-10-13T23:46:53.654 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-10-13T23:46:53.655 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-10-13T23:46:53.655 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2023-10-13T23:46:53.680 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2023-10-13T23:46:53.682 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-10-13T23:46:53.846 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-10-13T23:46:53.929 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2023-10-13T23:46:54.024 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2023-10-13T23:46:54.053 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-10-13T23:46:54.104 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [640 B] 2023-10-13T23:46:58.390 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 9,489 kB in 2s (3,903 kB/s) 2023-10-13T23:46:59.617 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:46:59.632 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-13T23:46:59.683 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:46:59.847 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2023-10-13T23:46:59.847 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2023-10-13T23:46:59.987 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2023-10-13T23:46:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-13T23:46:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-13T23:46:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-13T23:46:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-13T23:46:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: python2.7-minimal 2023-10-13T23:46:59.989 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-13T23:46:59.989 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2023-10-13T23:46:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-5.15.0-86 linux-headers-5.15.0-86-generic 2023-10-13T23:46:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-generic linux-image-5.15.0-86-generic 2023-10-13T23:46:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-5.15.0-86-generic linux-modules-extra-5.15.0-86-generic 2023-10-13T23:46:59.990 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2023-10-13T23:46:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-10-13T23:46:59.991 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2023-10-13T23:46:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: thermald 2023-10-13T23:47:00.044 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2023-10-13T23:47:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: linux-headers-5.15.0-86 linux-headers-5.15.0-86-generic 2023-10-13T23:47:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.15.0-86-generic linux-modules-5.15.0-86-generic 2023-10-13T23:47:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-extra-5.15.0-86-generic 2023-10-13T23:47:00.045 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages will be upgraded: 2023-10-13T23:47:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-13T23:47:00.097 INFO:teuthology.orchestra.run.smithi183.stdout:3 upgraded, 5 newly installed, 0 to remove and 268 not upgraded. 2023-10-13T23:47:00.098 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 113 MB of archives. 2023-10-13T23:47:00.098 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 582 MB of additional disk space will be used. 2023-10-13T23:47:00.098 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-86-generic amd64 5.15.0-86.96 [22.4 MB] 2023-10-13T23:47:00.557 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-86-generic amd64 5.15.0-86.96 [11.5 MB] 2023-10-13T23:47:00.661 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-86-generic amd64 5.15.0-86.96 [63.9 MB] 2023-10-13T23:47:01.543 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.86.83 [1,702 B] 2023-10-13T23:47:01.543 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.86.83 [2,472 B] 2023-10-13T23:47:01.543 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-86 all 5.15.0-86.96 [12.3 MB] 2023-10-13T23:47:01.668 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-86-generic amd64 5.15.0-86.96 [2,882 kB] 2023-10-13T23:47:01.690 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.86.83 [2,328 B] 2023-10-13T23:47:02.056 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 113 MB in 2s (69.0 MB/s) 2023-10-13T23:47:02.227 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-5.15.0-86-generic. 2023-10-13T23:47:04.212 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-10-13T23:47:04.217 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../0-linux-modules-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-13T23:47:04.308 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:06.459 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-image-5.15.0-86-generic. 2023-10-13T23:47:06.480 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../1-linux-image-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-13T23:47:06.565 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:07.029 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-86-generic. 2023-10-13T23:47:07.049 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-13T23:47:07.087 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-modules-extra-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:14.163 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../3-linux-generic_5.15.0.86.83_amd64.deb ... 2023-10-13T23:47:14.288 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-13T23:47:14.637 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.86.83_amd64.deb ... 2023-10-13T23:47:14.758 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-image-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-13T23:47:15.053 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-5.15.0-86. 2023-10-13T23:47:15.064 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../5-linux-headers-5.15.0-86_5.15.0-86.96_all.deb ... 2023-10-13T23:47:15.094 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-5.15.0-86 (5.15.0-86.96) ... 2023-10-13T23:47:21.027 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package linux-headers-5.15.0-86-generic. 2023-10-13T23:47:21.064 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../6-linux-headers-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-13T23:47:21.127 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:23.795 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.86.83_amd64.deb ... 2023-10-13T23:47:23.921 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking linux-headers-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-13T23:47:24.262 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-5.15.0-86 (5.15.0-86.96) ... 2023-10-13T23:47:24.384 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:24.535 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-headers-generic (5.15.0.86.83) ... 2023-10-13T23:47:24.662 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:25.991 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-10-13T23:47:25.992 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-10-13T23:47:25.992 INFO:teuthology.orchestra.run.smithi183.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-86-generic 2023-10-13T23:47:25.992 INFO:teuthology.orchestra.run.smithi183.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-86-generic 2023-10-13T23:47:26.174 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:27.675 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-modules-extra-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:29.060 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-image-generic (5.15.0.86.83) ... 2023-10-13T23:47:29.186 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up linux-generic (5.15.0.86.83) ... 2023-10-13T23:47:29.312 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-13T23:47:29.369 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-13T23:47:29.369 INFO:teuthology.orchestra.run.smithi183.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-86-generic 2023-10-13T23:47:44.272 INFO:teuthology.orchestra.run.smithi183.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-13T23:47:44.272 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub' 2023-10-13T23:47:44.300 INFO:teuthology.orchestra.run.smithi183.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-13T23:47:44.307 INFO:teuthology.orchestra.run.smithi183.stdout:Generating grub configuration file ... 2023-10-13T23:47:44.683 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.15.0-86-generic 2023-10-13T23:47:44.703 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.15.0-86-generic 2023-10-13T23:47:45.081 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-13T23:47:45.087 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-13T23:47:45.156 INFO:teuthology.orchestra.run.smithi183.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-13T23:47:45.161 INFO:teuthology.orchestra.run.smithi183.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-13T23:47:45.329 INFO:teuthology.orchestra.run.smithi183.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-13T23:47:45.329 INFO:teuthology.orchestra.run.smithi183.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-10-13T23:47:45.330 INFO:teuthology.orchestra.run.smithi183.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-13T23:47:45.382 INFO:teuthology.orchestra.run.smithi183.stdout:done 2023-10-13T23:47:45.865 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2023-10-13T23:47:45.889 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2023-10-13T23:47:45.889 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2023-10-13T23:47:45.889 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2023-10-13T23:47:45.889 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2023-10-13T23:47:45.890 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 20 2023-10-13T23:47:45.890 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2023-10-13T23:47:45.890 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2023-10-13T23:47:45.890 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2023-10-13T23:47:45.891 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 5.15.0.86.83 2023-10-13T23:47:45.891 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-86), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-13T23:47:45.891 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-5.15.0-86-generic, linux-modules-extra-5.15.0-86-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-13T23:47:45.891 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2023-10-13T23:47:45.891 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2023-10-13T23:47:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2023-10-13T23:47:45.892 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2023-10-13T23:47:45.892 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-86-generic 2023-10-13T23:47:45.892 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-13T23:47:45.893 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-13T23:47:45.893 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-13T23:47:45.893 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2023-10-13T23:47:45.893 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get clean 2023-10-13T23:47:46.078 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2023-10-13T23:47:46.336 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-13T23:47:46.337 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-13T23:47:46.408 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-13T23:47:46.490 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-13T23:47:47.614 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:47:47.631 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-13T23:47:47.682 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:47:47.846 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2023-10-13T23:47:47.847 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2023-10-13T23:47:47.987 INFO:teuthology.orchestra.run.smithi183.stdout:linux-image-generic is already the newest version (5.15.0.86.83). 2023-10-13T23:47:47.988 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2023-10-13T23:47:47.988 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-13T23:47:47.988 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-13T23:47:47.989 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-13T23:47:47.989 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-13T23:47:47.989 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-13T23:47:47.989 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-13T23:47:48.024 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 0 newly installed, 0 to remove and 268 not upgraded. 2023-10-13T23:47:48.026 DEBUG:teuthology.orchestra.run.smithi183:> dpkg -s linux-image-generic 2023-10-13T23:47:48.084 INFO:teuthology.orchestra.run.smithi183.stdout:Package: linux-image-generic 2023-10-13T23:47:48.084 INFO:teuthology.orchestra.run.smithi183.stdout:Status: install ok installed 2023-10-13T23:47:48.084 INFO:teuthology.orchestra.run.smithi183.stdout:Priority: optional 2023-10-13T23:47:48.084 INFO:teuthology.orchestra.run.smithi183.stdout:Section: kernel 2023-10-13T23:47:48.085 INFO:teuthology.orchestra.run.smithi183.stdout:Installed-Size: 20 2023-10-13T23:47:48.085 INFO:teuthology.orchestra.run.smithi183.stdout:Maintainer: Ubuntu Kernel Team 2023-10-13T23:47:48.085 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: amd64 2023-10-13T23:47:48.085 INFO:teuthology.orchestra.run.smithi183.stdout:Source: linux-meta 2023-10-13T23:47:48.085 INFO:teuthology.orchestra.run.smithi183.stdout:Version: 5.15.0.86.83 2023-10-13T23:47:48.086 INFO:teuthology.orchestra.run.smithi183.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-86), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-13T23:47:48.086 INFO:teuthology.orchestra.run.smithi183.stdout:Depends: linux-image-5.15.0-86-generic, linux-modules-extra-5.15.0-86-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-13T23:47:48.086 INFO:teuthology.orchestra.run.smithi183.stdout:Recommends: thermald 2023-10-13T23:47:48.086 INFO:teuthology.orchestra.run.smithi183.stdout:Description: Generic Linux kernel image 2023-10-13T23:47:48.086 INFO:teuthology.orchestra.run.smithi183.stdout: This package will always depend on the latest generic kernel image 2023-10-13T23:47:48.087 INFO:teuthology.orchestra.run.smithi183.stdout: available. 2023-10-13T23:47:48.087 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2023-10-13T23:47:48.131 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.pWCFJETAnx 2023-10-13T23:47:48.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub/grub.cfg /tmp/tmp.pWCFJETAnx 2023-10-13T23:47:48.184 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.pWCFJETAnx 2023-10-13T23:47:48.300 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.pWCFJETAnx is 10KB 2023-10-13T23:47:48.313 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.pWCFJETAnx 2023-10-13T23:47:48.318 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-13T23:47:48.373 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:47:48.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-13T23:47:48.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-13T23:47:48.449 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-86-generic 2023-10-13T23:47:48.449 DEBUG:teuthology.orchestra.run.smithi183:> sudo update-grub 2023-10-13T23:47:49.647 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub' 2023-10-13T23:47:49.649 INFO:teuthology.orchestra.run.smithi183.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-13T23:47:49.652 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2023-10-13T23:47:49.907 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.15.0-86-generic 2023-10-13T23:47:49.913 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.15.0-86-generic 2023-10-13T23:47:50.208 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-13T23:47:50.213 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-13T23:47:50.252 INFO:teuthology.orchestra.run.smithi183.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-13T23:47:50.256 INFO:teuthology.orchestra.run.smithi183.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-13T23:47:50.414 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-13T23:47:50.415 INFO:teuthology.orchestra.run.smithi183.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-13T23:47:50.415 INFO:teuthology.orchestra.run.smithi183.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-13T23:47:50.426 INFO:teuthology.orchestra.run.smithi183.stderr:done 2023-10-13T23:47:50.446 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2023-10-13T23:48:20.478 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-13T23:48:20.479 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-13T23:48:20.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T23:49:20.539 DEBUG:teuthology.orchestra.remote:timed out 2023-10-13T23:49:29.547 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-13T23:49:29.548 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T23:49:30.559 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2023-10-13T23:49:42.570 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-13T23:49:42.571 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T23:49:42.805 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-10-13T23:49:43.447 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-13T23:49:43.447 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-86-generic"... 2023-10-13T23:49:43.447 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2023-10-13T23:49:43.495 INFO:teuthology.orchestra.run.smithi183.stdout:5.15.0-86-generic 2023-10-13T23:49:43.495 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-86-generic vs 5.15.0-86-generic 2023-10-13T23:49:43.495 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-13T23:49:43.496 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-13T23:49:44.497 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-13T23:49:44.497 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-13T23:49:44.570 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2023-10-13T23:49:44.605 DEBUG:teuthology.parallel:result is None 2023-10-13T23:49:44.605 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-13T23:49:44.614 INFO:teuthology.task.internal:Creating test directory... 2023-10-13T23:49:44.614 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-13T23:49:44.620 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-13T23:49:44.656 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-13T23:49:44.723 INFO:teuthology.task.internal:Creating archive directory... 2023-10-13T23:49:44.724 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-13T23:49:44.742 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-13T23:49:44.774 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-13T23:49:44.774 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-13T23:49:44.801 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-13T23:49:44.809 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-13T23:49:44.811 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-13T23:49:44.823 INFO:teuthology.task.internal:Configuring sudo... 2023-10-13T23:49:44.823 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-13T23:49:44.866 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-13T23:49:44.875 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-13T23:49:44.876 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-13T23:49:44.915 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-13T23:49:44.963 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-13T23:49:45.011 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:49:45.011 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-13T23:49:45.070 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2023-10-13T23:49:45.182 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-13T23:49:45.189 INFO:teuthology.task.internal:Starting timer... 2023-10-13T23:49:45.190 INFO:teuthology.run_tasks:Running task pcp... 2023-10-13T23:49:45.218 INFO:teuthology.run_tasks:Running task selinux... 2023-10-13T23:49:45.267 DEBUG:teuthology.task.selinux:Excluding smithi183: OS 'ubuntu' does not support SELinux 2023-10-13T23:49:45.268 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-13T23:49:45.268 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-13T23:49:45.269 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-13T23:49:45.269 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-13T23:49:45.317 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-13T23:49:45.318 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-13T23:49:45.333 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-13T23:49:45.335 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-13T23:57:24.920 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2023-10-13T23:57:24.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-13T23:57:24.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-13T23:57:24.999 DEBUG:teuthology.orchestra.run.smithi183:> true 2023-10-13T23:57:25.103 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2023-10-13T23:57:25.103 INFO:teuthology.run_tasks:Running task clock... 2023-10-13T23:57:25.115 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-13T23:57:25.116 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-13T23:57:25.117 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-13T23:57:25.173 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-13T23:57:25.174 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Command line: ntpd -gq 2023-10-13T23:57:25.174 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: ---------------------------------------------------- 2023-10-13T23:57:25.174 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: ntp-4 is maintained by Network Time Foundation, 2023-10-13T23:57:25.174 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-13T23:57:25.175 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: corporation. Support and training for ntp-4 are 2023-10-13T23:57:25.175 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: available at https://www.nwtime.org/support 2023-10-13T23:57:25.175 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: ---------------------------------------------------- 2023-10-13T23:57:25.176 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: proto: precision = 0.043 usec (-24) 2023-10-13T23:57:25.176 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: basedate set to 2022-02-04 2023-10-13T23:57:25.176 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: gps base set to 2022-02-06 (week 2196) 2023-10-13T23:57:25.176 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-13T23:57:25.177 INFO:teuthology.orchestra.run.smithi183.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-13T23:57:25.177 INFO:teuthology.orchestra.run.smithi183.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-13T23:57:25.178 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: restrict ::: KOD does nothing without LIMITED. 2023-10-13T23:57:25.178 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Listen and drop on 0 v6wildcard [::]:123 2023-10-13T23:57:25.179 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-13T23:57:25.179 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Listen normally on 2 lo 127.0.0.1:123 2023-10-13T23:57:25.179 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Listen normally on 3 enp3s0f1 172.21.15.183:123 2023-10-13T23:57:25.179 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Listen normally on 4 lo [::1]:123 2023-10-13T23:57:25.179 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8085%5]:123 2023-10-13T23:57:25.180 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:25 ntpd[27985]: Listening on routing socket on fd #22 for interface updates 2023-10-13T23:57:27.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:27 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:27.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:27 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:27.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:27 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:27.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:27 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:27.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:27 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:27.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:27 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:28.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:28 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:28.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:28 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:28.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:28 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:28.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:28 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:28.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:28 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:28.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:28 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:29.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:29 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:29.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:29 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:29.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:29 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:29.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:29 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:29.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:29 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:29.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:29 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:30.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:30.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:30.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:30.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:30.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:30.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:30.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:30.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:30.177 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:30.177 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:30.177 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:30.177 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:30.177 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:30.178 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:30 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:31.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:31 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:31.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:31 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:31.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:31 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:31.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:31 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:31.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:31 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:31.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:31 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:32.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:32.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:32.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:32.177 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:32 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:33.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:33.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:33.174 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-13T23:57:33.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: can't open /var/log/ntpstats/rawstats.20231013: Permission denied 2023-10-13T23:57:33.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-13T23:57:33.175 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: can't open /var/log/ntpstats/peerstats.20231013: Permission denied 2023-10-13T23:57:33.175 INFO:teuthology.orchestra.run.smithi183.stdout:13 Oct 23:57:33 ntpd[27985]: ntpd: time slew -0.000629 s 2023-10-13T23:57:33.176 INFO:teuthology.orchestra.run.smithi183.stdout:ntpd: time slew -0.000629s 2023-10-13T23:57:33.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-13T23:57:33.176 INFO:teuthology.orchestra.run.smithi183.stderr:13 Oct 23:57:33 ntpd[27985]: can't open /var/log/ntpstats/loopstats.20231013: Permission denied 2023-10-13T23:57:33.251 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2023-10-13T23:57:33.251 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2023-10-13T23:57:33.252 INFO:teuthology.orchestra.run.smithi183.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-13T23:57:33.252 INFO:teuthology.orchestra.run.smithi183.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-13T23:57:33.252 INFO:teuthology.orchestra.run.smithi183.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-13T23:57:33.252 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-13T23:57:33.253 INFO:teuthology.run_tasks:Running task install... 2023-10-13T23:57:33.262 DEBUG:teuthology.task.install:project ceph 2023-10-13T23:57:33.263 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '01fff6a72a328459c1d153e5dc1de6a34e48a82f'}} 2023-10-13T23:57:33.263 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '01fff6a72a328459c1d153e5dc1de6a34e48a82f'} 2023-10-13T23:57:33.263 INFO:teuthology.task.install:Using flavor: default 2023-10-13T23:57:33.269 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-13T23:57:33.269 INFO:teuthology.task.install:extra packages: [] 2023-10-13T23:57:33.270 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-key list | grep Ceph 2023-10-13T23:57:33.350 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-13T23:57:33.378 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-13T23:57:33.378 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph.com (release key) 2023-10-13T23:57:33.379 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-10-13T23:57:33.379 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=01fff6a72a328459c1d153e5dc1de6a34e48a82f 2023-10-13T23:57:33.601 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default/ 2023-10-13T23:57:33.601 INFO:teuthology.task.install.deb:Package version is 18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:57:33.794 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:57:33.794 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-13T23:57:33.808 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2023-10-13T23:57:33.975 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-13T23:57:33.975 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-13T23:57:33.993 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-13T23:57:34.013 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-13T23:57:34.179 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy InRelease 2023-10-13T23:57:34.240 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-13T23:57:34.301 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-13T23:57:34.362 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,411 B] 2023-10-13T23:57:34.431 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-10-13T23:57:34.527 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 28.5 kB in 1s (54.7 kB/s) 2023-10-13T23:57:35.418 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:57:35.433 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6724-g01fff6a7-1jammy cephadm=18.0.0-6724-g01fff6a7-1jammy ceph-mds=18.0.0-6724-g01fff6a7-1jammy ceph-mgr=18.0.0-6724-g01fff6a7-1jammy ceph-common=18.0.0-6724-g01fff6a7-1jammy ceph-fuse=18.0.0-6724-g01fff6a7-1jammy ceph-test=18.0.0-6724-g01fff6a7-1jammy ceph-volume=18.0.0-6724-g01fff6a7-1jammy radosgw=18.0.0-6724-g01fff6a7-1jammy python3-rados=18.0.0-6724-g01fff6a7-1jammy python3-rgw=18.0.0-6724-g01fff6a7-1jammy python3-cephfs=18.0.0-6724-g01fff6a7-1jammy python3-rbd=18.0.0-6724-g01fff6a7-1jammy libcephfs2=18.0.0-6724-g01fff6a7-1jammy libcephfs-dev=18.0.0-6724-g01fff6a7-1jammy librados2=18.0.0-6724-g01fff6a7-1jammy librbd1=18.0.0-6724-g01fff6a7-1jammy rbd-fuse=18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:57:35.493 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:57:35.678 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2023-10-13T23:57:35.679 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2023-10-13T23:57:35.846 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2023-10-13T23:57:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-13T23:57:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-13T23:57:35.846 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-13T23:57:35.847 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-13T23:57:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-13T23:57:35.848 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-13T23:57:35.848 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2023-10-13T23:57:35.848 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-13T23:57:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-13T23:57:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-13T23:57:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-13T23:57:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-13T23:57:35.849 INFO:teuthology.orchestra.run.smithi183.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-13T23:57:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-13T23:57:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-13T23:57:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2023-10-13T23:57:35.850 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-13T23:57:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-13T23:57:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-13T23:57:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-13T23:57:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-13T23:57:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-13T23:57:35.851 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2023-10-13T23:57:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2023-10-13T23:57:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-13T23:57:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-13T23:57:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-13T23:57:35.852 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-13T23:57:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat xmlstarlet 2023-10-13T23:57:35.853 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2023-10-13T23:57:35.853 INFO:teuthology.orchestra.run.smithi183.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-13T23:57:35.854 INFO:teuthology.orchestra.run.smithi183.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-13T23:57:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-13T23:57:35.855 INFO:teuthology.orchestra.run.smithi183.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-13T23:57:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-13T23:57:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-13T23:57:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: python3-watchdog 2023-10-13T23:57:35.856 INFO:teuthology.orchestra.run.smithi183.stdout:Recommended packages: 2023-10-13T23:57:35.856 INFO:teuthology.orchestra.run.smithi183.stdout: btrfs-tools 2023-10-13T23:57:35.947 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2023-10-13T23:57:35.948 INFO:teuthology.orchestra.run.smithi183.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-13T23:57:35.948 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-13T23:57:35.948 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2023-10-13T23:57:35.949 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 2023-10-13T23:57:35.949 INFO:teuthology.orchestra.run.smithi183.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2023-10-13T23:57:35.950 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-13T23:57:35.950 INFO:teuthology.orchestra.run.smithi183.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-13T23:57:35.950 INFO:teuthology.orchestra.run.smithi183.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-13T23:57:35.950 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-13T23:57:35.950 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-google-auth 2023-10-13T23:57:35.951 INFO:teuthology.orchestra.run.smithi183.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-13T23:57:35.951 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-13T23:57:35.951 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-13T23:57:35.952 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-13T23:57:35.952 INFO:teuthology.orchestra.run.smithi183.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-13T23:57:35.952 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-13T23:57:35.952 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-13T23:57:35.952 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-13T23:57:35.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-13T23:57:35.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-13T23:57:35.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-13T23:57:35.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-13T23:57:35.953 INFO:teuthology.orchestra.run.smithi183.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-13T23:57:35.954 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-13T23:57:35.954 INFO:teuthology.orchestra.run.smithi183.stdout: socat xmlstarlet 2023-10-13T23:57:36.137 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 109 newly installed, 0 to remove and 256 not upgraded. 2023-10-13T23:57:36.137 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 184 MB of archives. 2023-10-13T23:57:36.137 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 824 MB of additional disk space will be used. 2023-10-13T23:57:36.137 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-13T23:57:36.192 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6724-g01fff6a7-1jammy [3,450 kB] 2023-10-13T23:57:36.410 INFO:teuthology.orchestra.run.smithi183.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-13T23:57:36.497 INFO:teuthology.orchestra.run.smithi183.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6724-g01fff6a7-1jammy [3,130 kB] 2023-10-13T23:57:36.581 INFO:teuthology.orchestra.run.smithi183.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-13T23:57:36.592 INFO:teuthology.orchestra.run.smithi183.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6724-g01fff6a7-1jammy [688 kB] 2023-10-13T23:57:36.595 INFO:teuthology.orchestra.run.smithi183.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-13T23:57:36.614 INFO:teuthology.orchestra.run.smithi183.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6724-g01fff6a7-1jammy [356 kB] 2023-10-13T23:57:36.626 INFO:teuthology.orchestra.run.smithi183.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6724-g01fff6a7-1jammy [32.8 kB] 2023-10-13T23:57:36.626 INFO:teuthology.orchestra.run.smithi183.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6724-g01fff6a7-1jammy [180 kB] 2023-10-13T23:57:36.632 INFO:teuthology.orchestra.run.smithi183.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6724-g01fff6a7-1jammy [62.4 kB] 2023-10-13T23:57:36.634 INFO:teuthology.orchestra.run.smithi183.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6724-g01fff6a7-1jammy [330 kB] 2023-10-13T23:57:36.645 INFO:teuthology.orchestra.run.smithi183.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6724-g01fff6a7-1jammy [6,222 kB] 2023-10-13T23:57:36.667 INFO:teuthology.orchestra.run.smithi183.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-13T23:57:36.853 INFO:teuthology.orchestra.run.smithi183.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6724-g01fff6a7-1jammy [112 kB] 2023-10-13T23:57:36.855 INFO:teuthology.orchestra.run.smithi183.stdout:Get:16 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6724-g01fff6a7-1jammy [439 kB] 2023-10-13T23:57:36.861 INFO:teuthology.orchestra.run.smithi183.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-13T23:57:36.867 INFO:teuthology.orchestra.run.smithi183.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6724-g01fff6a7-1jammy [22.6 MB] 2023-10-13T23:57:36.871 INFO:teuthology.orchestra.run.smithi183.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-13T23:57:36.902 INFO:teuthology.orchestra.run.smithi183.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-13T23:57:36.908 INFO:teuthology.orchestra.run.smithi183.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-13T23:57:36.909 INFO:teuthology.orchestra.run.smithi183.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-13T23:57:36.911 INFO:teuthology.orchestra.run.smithi183.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-13T23:57:37.092 INFO:teuthology.orchestra.run.smithi183.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-13T23:57:37.304 INFO:teuthology.orchestra.run.smithi183.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-13T23:57:37.318 INFO:teuthology.orchestra.run.smithi183.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-13T23:57:37.436 INFO:teuthology.orchestra.run.smithi183.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-13T23:57:37.444 INFO:teuthology.orchestra.run.smithi183.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-13T23:57:37.455 INFO:teuthology.orchestra.run.smithi183.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-13T23:57:37.471 INFO:teuthology.orchestra.run.smithi183.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-13T23:57:37.485 INFO:teuthology.orchestra.run.smithi183.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-13T23:57:37.490 INFO:teuthology.orchestra.run.smithi183.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-13T23:57:37.494 INFO:teuthology.orchestra.run.smithi183.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-13T23:57:37.597 INFO:teuthology.orchestra.run.smithi183.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-13T23:57:37.716 INFO:teuthology.orchestra.run.smithi183.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6724-g01fff6a7-1jammy [6,853 kB] 2023-10-13T23:57:37.770 INFO:teuthology.orchestra.run.smithi183.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-13T23:57:37.869 INFO:teuthology.orchestra.run.smithi183.stdout:Get:37 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6724-g01fff6a7-1jammy [238 kB] 2023-10-13T23:57:37.875 INFO:teuthology.orchestra.run.smithi183.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6724-g01fff6a7-1jammy [128 kB] 2023-10-13T23:57:37.879 INFO:teuthology.orchestra.run.smithi183.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6724-g01fff6a7-1jammy [1,496 kB] 2023-10-13T23:57:37.928 INFO:teuthology.orchestra.run.smithi183.stdout:Get:40 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6724-g01fff6a7-1jammy [9,863 kB] 2023-10-13T23:57:37.937 INFO:teuthology.orchestra.run.smithi183.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-13T23:57:37.958 INFO:teuthology.orchestra.run.smithi183.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-13T23:57:38.093 INFO:teuthology.orchestra.run.smithi183.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-13T23:57:38.110 INFO:teuthology.orchestra.run.smithi183.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-13T23:57:38.118 INFO:teuthology.orchestra.run.smithi183.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-13T23:57:38.191 INFO:teuthology.orchestra.run.smithi183.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-13T23:57:38.218 INFO:teuthology.orchestra.run.smithi183.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-13T23:57:38.226 INFO:teuthology.orchestra.run.smithi183.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-13T23:57:38.254 INFO:teuthology.orchestra.run.smithi183.stdout:Get:49 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6724-g01fff6a7-1jammy [31.7 MB] 2023-10-13T23:57:38.349 INFO:teuthology.orchestra.run.smithi183.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-13T23:57:38.350 INFO:teuthology.orchestra.run.smithi183.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-13T23:57:38.517 INFO:teuthology.orchestra.run.smithi183.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-13T23:57:38.706 INFO:teuthology.orchestra.run.smithi183.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-13T23:57:38.860 INFO:teuthology.orchestra.run.smithi183.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-13T23:57:38.965 INFO:teuthology.orchestra.run.smithi183.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-13T23:57:38.987 INFO:teuthology.orchestra.run.smithi183.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-13T23:57:39.018 INFO:teuthology.orchestra.run.smithi183.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-13T23:57:39.031 INFO:teuthology.orchestra.run.smithi183.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-13T23:57:39.071 INFO:teuthology.orchestra.run.smithi183.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-13T23:57:39.080 INFO:teuthology.orchestra.run.smithi183.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-13T23:57:39.109 INFO:teuthology.orchestra.run.smithi183.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-13T23:57:39.376 INFO:teuthology.orchestra.run.smithi183.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-13T23:57:39.445 INFO:teuthology.orchestra.run.smithi183.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6724-g01fff6a7-1jammy [14.0 kB] 2023-10-13T23:57:39.446 INFO:teuthology.orchestra.run.smithi183.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6724-g01fff6a7-1jammy [839 kB] 2023-10-13T23:57:39.463 INFO:teuthology.orchestra.run.smithi183.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6724-g01fff6a7-1jammy [2,146 kB] 2023-10-13T23:57:39.500 INFO:teuthology.orchestra.run.smithi183.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6724-g01fff6a7-1jammy [283 kB] 2023-10-13T23:57:39.504 INFO:teuthology.orchestra.run.smithi183.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6724-g01fff6a7-1jammy [138 kB] 2023-10-13T23:57:39.507 INFO:teuthology.orchestra.run.smithi183.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6724-g01fff6a7-1jammy [1,906 kB] 2023-10-13T23:57:39.527 INFO:teuthology.orchestra.run.smithi183.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6724-g01fff6a7-1jammy [8,625 kB] 2023-10-13T23:57:39.541 INFO:teuthology.orchestra.run.smithi183.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-13T23:57:39.806 INFO:teuthology.orchestra.run.smithi183.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6724-g01fff6a7-1jammy [14.2 kB] 2023-10-13T23:57:39.807 INFO:teuthology.orchestra.run.smithi183.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6724-g01fff6a7-1jammy [47.8 MB] 2023-10-13T23:57:39.891 INFO:teuthology.orchestra.run.smithi183.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-13T23:57:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-13T23:57:39.915 INFO:teuthology.orchestra.run.smithi183.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-13T23:57:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-13T23:57:40.274 INFO:teuthology.orchestra.run.smithi183.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-13T23:57:40.284 INFO:teuthology.orchestra.run.smithi183.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-13T23:57:40.285 INFO:teuthology.orchestra.run.smithi183.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-13T23:57:40.350 INFO:teuthology.orchestra.run.smithi183.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-13T23:57:40.351 INFO:teuthology.orchestra.run.smithi183.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-13T23:57:40.353 INFO:teuthology.orchestra.run.smithi183.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-13T23:57:40.517 INFO:teuthology.orchestra.run.smithi183.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-13T23:57:40.707 INFO:teuthology.orchestra.run.smithi183.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-13T23:57:40.770 INFO:teuthology.orchestra.run.smithi183.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-13T23:57:40.806 INFO:teuthology.orchestra.run.smithi183.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-13T23:57:40.864 INFO:teuthology.orchestra.run.smithi183.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-13T23:57:41.035 INFO:teuthology.orchestra.run.smithi183.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-13T23:57:41.073 INFO:teuthology.orchestra.run.smithi183.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-13T23:57:41.096 INFO:teuthology.orchestra.run.smithi183.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-13T23:57:41.104 INFO:teuthology.orchestra.run.smithi183.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-13T23:57:41.146 INFO:teuthology.orchestra.run.smithi183.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-13T23:57:41.169 INFO:teuthology.orchestra.run.smithi183.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-13T23:57:41.368 INFO:teuthology.orchestra.run.smithi183.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-13T23:57:41.596 INFO:teuthology.orchestra.run.smithi183.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6724-g01fff6a7-1jammy [120 kB] 2023-10-13T23:57:41.598 INFO:teuthology.orchestra.run.smithi183.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6724-g01fff6a7-1jammy [38.3 kB] 2023-10-13T23:57:41.599 INFO:teuthology.orchestra.run.smithi183.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6724-g01fff6a7-1jammy [12.3 MB] 2023-10-13T23:57:41.851 INFO:teuthology.orchestra.run.smithi183.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-13T23:57:41.852 INFO:teuthology.orchestra.run.smithi183.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-13T23:57:41.861 INFO:teuthology.orchestra.run.smithi183.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-13T23:57:41.864 INFO:teuthology.orchestra.run.smithi183.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-13T23:57:41.895 INFO:teuthology.orchestra.run.smithi183.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-13T23:57:41.908 INFO:teuthology.orchestra.run.smithi183.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-13T23:57:41.909 INFO:teuthology.orchestra.run.smithi183.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6724-g01fff6a7-1jammy [91.4 kB] 2023-10-13T23:57:42.038 INFO:teuthology.orchestra.run.smithi183.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-13T23:57:42.039 INFO:teuthology.orchestra.run.smithi183.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-13T23:57:42.041 INFO:teuthology.orchestra.run.smithi183.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-13T23:57:42.059 INFO:teuthology.orchestra.run.smithi183.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-13T23:57:42.229 INFO:teuthology.orchestra.run.smithi183.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-13T23:57:43.404 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 184 MB in 7s (26.6 MB/s) 2023-10-13T23:57:43.561 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-13T23:57:43.661 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2023-10-13T23:57:43.667 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../000-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-13T23:57:43.751 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-13T23:57:44.071 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-13T23:57:44.098 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../001-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-13T23:57:44.137 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-13T23:57:44.482 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-13T23:57:44.509 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../002-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-13T23:57:44.549 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-13T23:57:44.827 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-13T23:57:44.853 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../003-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-13T23:57:44.885 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-13T23:57:45.322 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-13T23:57:45.349 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../004-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-13T23:57:45.397 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-13T23:57:45.825 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-13T23:57:45.851 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../005-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-13T23:57:45.891 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-13T23:57:46.169 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-13T23:57:46.184 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../006-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-13T23:57:46.219 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-13T23:57:46.547 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-13T23:57:46.573 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../007-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-13T23:57:46.614 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-13T23:57:46.941 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librados2. 2023-10-13T23:57:46.968 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../008-librados2_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:47.009 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librados2 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:47.410 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librbd1. 2023-10-13T23:57:47.426 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../009-librbd1_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:47.460 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librbd1 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:47.849 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs2. 2023-10-13T23:57:47.875 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../010-libcephfs2_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:47.915 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs2 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:48.201 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rados. 2023-10-13T23:57:48.227 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../011-python3-rados_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:48.267 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rados (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:48.754 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-13T23:57:48.780 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../012-python3-ceph-argparse_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:57:48.820 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-argparse (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:49.073 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cephfs. 2023-10-13T23:57:49.099 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../013-python3-cephfs_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:49.139 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cephfs (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:49.434 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-13T23:57:49.460 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../014-python3-ceph-common_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:57:49.500 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-ceph-common (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:49.828 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-13T23:57:49.855 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-13T23:57:49.895 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-13T23:57:50.198 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-prettytable. 2023-10-13T23:57:50.225 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2023-10-13T23:57:50.264 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-13T23:57:50.517 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rbd. 2023-10-13T23:57:50.532 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../017-python3-rbd_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:50.566 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rbd (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:50.861 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-13T23:57:50.888 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-13T23:57:50.928 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-13T23:57:51.281 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package librgw2. 2023-10-13T23:57:51.309 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../019-librgw2_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:51.347 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking librgw2 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:51.841 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rgw. 2023-10-13T23:57:51.868 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../020-python3-rgw_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:51.908 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rgw (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:52.227 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-13T23:57:52.254 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../021-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-13T23:57:52.285 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-13T23:57:52.580 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libradosstriper1. 2023-10-13T23:57:52.607 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../022-libradosstriper1_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:52.647 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libradosstriper1 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:53.050 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-common. 2023-10-13T23:57:53.077 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../023-ceph-common_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:53.116 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-common (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:54.447 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-base. 2023-10-13T23:57:54.473 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../024-ceph-base_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:57:54.523 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-base (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:57:55.193 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-13T23:57:55.220 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../025-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-13T23:57:55.260 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-13T23:57:55.562 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cheroot. 2023-10-13T23:57:55.589 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../026-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-13T23:57:55.637 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-13T23:57:55.957 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-13T23:57:55.984 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../027-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-13T23:57:56.024 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-13T23:57:56.318 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-13T23:57:56.345 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../028-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-13T23:57:56.384 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-13T23:57:56.654 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-13T23:57:56.680 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../029-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-13T23:57:56.720 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-13T23:57:57.023 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempora. 2023-10-13T23:57:57.050 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../030-python3-tempora_4.1.2-1_all.deb ... 2023-10-13T23:57:57.090 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-13T23:57:57.393 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-portend. 2023-10-13T23:57:57.419 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../031-python3-portend_3.0.0-1_all.deb ... 2023-10-13T23:57:57.459 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-13T23:57:57.712 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-13T23:57:57.727 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../032-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-13T23:57:57.762 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-13T23:57:58.048 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-13T23:57:58.075 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../033-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-13T23:57:58.115 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-13T23:57:58.451 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-natsort. 2023-10-13T23:57:58.478 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../034-python3-natsort_8.0.2-1_all.deb ... 2023-10-13T23:57:58.509 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-13T23:57:58.796 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-logutils. 2023-10-13T23:57:58.823 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../035-python3-logutils_0.3.3-8_all.deb ... 2023-10-13T23:57:58.862 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-13T23:57:59.349 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-13T23:57:59.375 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../036-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-13T23:57:59.415 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-13T23:57:59.701 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-mako. 2023-10-13T23:57:59.728 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../037-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-13T23:57:59.768 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-13T23:58:00.088 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-13T23:58:00.115 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../038-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-13T23:58:00.154 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-13T23:58:00.416 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-13T23:58:00.442 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../039-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-13T23:58:00.482 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-13T23:58:00.776 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webob. 2023-10-13T23:58:00.803 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../040-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-13T23:58:00.843 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-13T23:58:01.196 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-waitress. 2023-10-13T23:58:01.223 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../041-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-13T23:58:01.266 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-13T23:58:01.582 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-tempita. 2023-10-13T23:58:01.609 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-13T23:58:01.648 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-13T23:58:01.910 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-paste. 2023-10-13T23:58:01.936 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../043-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-13T23:58:01.968 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-13T23:58:02.296 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-13T23:58:02.312 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-13T23:58:02.354 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-13T23:58:02.624 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-13T23:58:02.650 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-13T23:58:02.682 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-13T23:58:02.985 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-webtest. 2023-10-13T23:58:03.000 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../046-python3-webtest_2.0.35-1_all.deb ... 2023-10-13T23:58:03.043 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-13T23:58:03.354 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pecan. 2023-10-13T23:58:03.381 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-13T23:58:03.421 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-13T23:58:03.776 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-certifi. 2023-10-13T23:58:03.803 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../048-python3-certifi_2020.6.20-1_all.deb ... 2023-10-13T23:58:03.842 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-13T23:58:04.170 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-idna. 2023-10-13T23:58:04.197 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../049-python3-idna_3.3-1_all.deb ... 2023-10-13T23:58:04.236 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-13T23:58:04.522 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-urllib3. 2023-10-13T23:58:04.549 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../050-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-13T23:58:04.589 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-13T23:58:04.876 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-requests. 2023-10-13T23:58:04.903 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../051-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-13T23:58:04.942 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-13T23:58:05.253 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-13T23:58:05.280 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../052-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-13T23:58:05.320 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-13T23:58:05.590 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-13T23:58:05.616 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../053-ceph-mgr-modules-core_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:58:05.665 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:06.044 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-13T23:58:06.071 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../054-libsqlite3-mod-ceph_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:06.111 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:06.405 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-openssl. 2023-10-13T23:58:06.433 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../055-python3-openssl_21.0.0-1_all.deb ... 2023-10-13T23:58:06.465 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-13T23:58:06.792 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr. 2023-10-13T23:58:06.818 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../056-ceph-mgr_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:06.858 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:07.244 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mon. 2023-10-13T23:58:07.271 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../057-ceph-mon_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:07.311 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mon (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:08.189 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-osd. 2023-10-13T23:58:08.216 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../058-ceph-osd_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:08.247 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-osd (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:10.642 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph. 2023-10-13T23:58:10.669 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../059-ceph_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:10.709 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:11.028 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-fuse. 2023-10-13T23:58:11.055 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../060-ceph-fuse_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:11.094 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-fuse (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:11.448 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mds. 2023-10-13T23:58:11.475 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../061-ceph-mds_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:11.523 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mds (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:11.884 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package cephadm. 2023-10-13T23:58:11.911 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../062-cephadm_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:11.951 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking cephadm (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:12.245 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-13T23:58:12.272 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../063-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-13T23:58:12.311 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-13T23:58:12.614 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-13T23:58:12.630 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../064-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-13T23:58:12.673 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-13T23:58:13.628 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-babel. 2023-10-13T23:58:13.655 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../065-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-13T23:58:13.694 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-13T23:58:13.997 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-jinja2. 2023-10-13T23:58:14.024 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../066-python3-jinja2_3.0.3-1_all.deb ... 2023-10-13T23:58:14.055 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-13T23:58:14.401 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-13T23:58:14.417 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../067-ceph-mgr-cephadm_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:58:14.450 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:14.762 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-13T23:58:14.778 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../068-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-13T23:58:14.829 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-13T23:58:15.090 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-13T23:58:15.117 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../069-python3-repoze.lru_0.7-2_all.deb ... 2023-10-13T23:58:15.165 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-13T23:58:15.493 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-routes. 2023-10-13T23:58:15.520 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../070-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-13T23:58:15.559 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-13T23:58:15.889 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-13T23:58:15.914 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../071-ceph-mgr-dashboard_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:58:15.954 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:16.332 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-13T23:58:16.357 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../072-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-13T23:58:16.399 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-13T23:58:16.861 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-joblib. 2023-10-13T23:58:16.887 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../073-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-13T23:58:16.927 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-13T23:58:17.255 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-13T23:58:17.271 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../074-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-13T23:58:17.305 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-13T23:58:17.600 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-sklearn. 2023-10-13T23:58:17.628 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../075-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-13T23:58:17.666 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-13T23:58:18.292 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-13T23:58:18.308 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../076-ceph-mgr-diskprediction-local_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:58:18.341 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:19.255 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-cachetools. 2023-10-13T23:58:19.280 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../077-python3-cachetools_5.0.0-1_all.deb ... 2023-10-13T23:58:19.319 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-13T23:58:19.622 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-13T23:58:19.649 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../078-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-13T23:58:19.688 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-13T23:58:20.209 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-13T23:58:20.236 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../079-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-13T23:58:20.275 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-13T23:58:20.603 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-rsa. 2023-10-13T23:58:20.630 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../080-python3-rsa_4.8-1_all.deb ... 2023-10-13T23:58:20.669 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-13T23:58:20.931 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-google-auth. 2023-10-13T23:58:20.958 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../081-python3-google-auth_1.5.1-3_all.deb ... 2023-10-13T23:58:20.989 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-13T23:58:21.292 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-13T23:58:21.320 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../082-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-13T23:58:21.359 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-13T23:58:21.780 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-websocket. 2023-10-13T23:58:21.807 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../083-python3-websocket_1.2.3-1_all.deb ... 2023-10-13T23:58:21.846 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-13T23:58:22.167 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-13T23:58:22.194 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../084-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-13T23:58:22.233 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-13T23:58:22.880 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-13T23:58:22.896 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../085-ceph-mgr-k8sevents_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:58:22.938 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:23.250 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-13T23:58:23.277 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../086-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-13T23:58:23.316 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-13T23:58:23.627 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-13T23:58:23.655 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../087-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-13T23:58:23.694 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-13T23:58:23.930 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package jq. 2023-10-13T23:58:23.957 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../088-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-13T23:58:23.996 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-13T23:58:24.257 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package socat. 2023-10-13T23:58:24.273 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../089-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-13T23:58:24.316 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-13T23:58:24.619 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package xmlstarlet. 2023-10-13T23:58:24.635 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../090-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-13T23:58:24.677 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-13T23:58:24.998 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-test. 2023-10-13T23:58:25.025 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../091-ceph-test_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:25.056 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-test (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:27.293 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package ceph-volume. 2023-10-13T23:58:27.309 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../092-ceph-volume_18.0.0-6724-g01fff6a7-1jammy_all.deb ... 2023-10-13T23:58:27.343 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking ceph-volume (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:27.580 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-13T23:58:27.596 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../093-libcephfs-dev_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:27.630 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libcephfs-dev (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:27.985 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package nvme-cli. 2023-10-13T23:58:28.012 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../094-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-13T23:58:28.051 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-13T23:58:28.398 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-13T23:58:28.426 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../095-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-13T23:58:28.465 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-13T23:58:28.876 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-13T23:58:28.904 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../096-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-13T23:58:28.943 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-13T23:58:29.270 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pastescript. 2023-10-13T23:58:29.298 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../097-python3-pastescript_2.0.2-4_all.deb ... 2023-10-13T23:58:29.337 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-13T23:58:29.624 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pluggy. 2023-10-13T23:58:29.651 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../098-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-13T23:58:29.682 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-13T23:58:29.968 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-psutil. 2023-10-13T23:58:29.995 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../099-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-13T23:58:30.035 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-13T23:58:30.613 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-py. 2023-10-13T23:58:30.640 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../100-python3-py_1.10.0-1_all.deb ... 2023-10-13T23:58:30.679 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-13T23:58:31.007 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pygments. 2023-10-13T23:58:31.035 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../101-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-13T23:58:31.074 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-13T23:58:31.545 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-13T23:58:31.573 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../102-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-13T23:58:31.611 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-13T23:58:31.897 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-toml. 2023-10-13T23:58:31.925 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../103-python3-toml_0.10.2-1_all.deb ... 2023-10-13T23:58:31.963 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-13T23:58:32.266 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-pytest. 2023-10-13T23:58:32.294 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../104-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-13T23:58:32.341 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-13T23:58:32.653 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package python3-simplejson. 2023-10-13T23:58:32.681 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../105-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-13T23:58:32.719 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-13T23:58:32.989 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-13T23:58:33.016 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../106-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-13T23:58:33.055 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-13T23:58:33.776 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package radosgw. 2023-10-13T23:58:33.804 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../107-radosgw_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:33.842 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking radosgw (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:34.595 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package rbd-fuse. 2023-10-13T23:58:34.623 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../108-rbd-fuse_18.0.0-6724-g01fff6a7-1jammy_amd64.deb ... 2023-10-13T23:58:34.662 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking rbd-fuse (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:34.940 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-13T23:58:35.216 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-13T23:58:35.351 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-13T23:58:35.613 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-13T23:58:36.002 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-13T23:58:36.467 INFO:teuthology.orchestra.run.smithi183.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-13T23:58:36.655 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up cephadm (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:37.441 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user cephadm....done 2023-10-13T23:58:37.667 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-13T23:58:37.935 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-13T23:58:38.195 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-13T23:58:38.321 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-13T23:58:38.580 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-13T23:58:38.831 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-13T23:58:38.974 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-13T23:58:39.268 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-13T23:58:39.627 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-13T23:58:39.978 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-13T23:58:40.404 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-13T23:58:40.789 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-13T23:58:41.206 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-13T23:58:41.457 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-argparse (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:41.717 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-13T23:58:41.818 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-13T23:58:42.069 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-13T23:58:42.195 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-13T23:58:42.554 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-13T23:58:42.680 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-13T23:58:42.956 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-13T23:58:43.249 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-13T23:58:43.559 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-13T23:58:44.227 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-13T23:58:44.486 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-13T23:58:44.612 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-13T23:58:44.738 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-13T23:58:45.031 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-13T23:58:45.399 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-13T23:58:45.641 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-13T23:58:46.002 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-13T23:58:46.294 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-13T23:58:46.637 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-13T23:58:46.888 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-13T23:58:46.998 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-ceph-common (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:47.274 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-idna (3.3-1) ... 2023-10-13T23:58:47.592 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-13T23:58:48.780 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-13T23:58:48.914 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-13T23:58:49.157 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-13T23:58:49.292 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-13T23:58:49.418 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-13T23:58:49.719 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-13T23:58:49.978 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-13T23:58:50.238 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-13T23:58:50.347 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-13T23:58:50.448 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-13T23:58:50.741 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-13T23:58:50.992 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-13T23:58:51.285 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-13T23:58:51.545 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-13T23:58:51.796 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-13T23:58:52.063 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-13T23:58:52.189 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-13T23:58:52.474 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-13T23:58:52.583 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-13T23:58:52.867 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-13T23:58:53.160 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-13T23:58:53.420 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-13T23:58:53.851 INFO:teuthology.orchestra.run.smithi183.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-13T23:58:54.106 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-13T23:58:54.223 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-13T23:58:54.608 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-13T23:58:54.867 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-13T23:58:55.127 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-13T23:58:55.261 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-13T23:58:55.579 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-13T23:58:55.856 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-13T23:58:56.131 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-13T23:58:56.257 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-13T23:58:56.525 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-13T23:58:56.818 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-13T23:58:57.077 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-13T23:58:57.453 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-13T23:58:57.738 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-13T23:58:58.057 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-13T23:58:58.183 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librados2 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:58.309 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librgw2 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:58.435 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:58.561 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs2 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:58.686 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libradosstriper1 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:58.881 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-13T23:58:59.158 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up librbd1 (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:59.284 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-modules-core (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:59.410 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-fuse (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:58:59.623 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-13T23:58:59.623 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-13T23:59:00.111 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libcephfs-dev (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:00.237 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rados (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:00.355 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-13T23:59:01.501 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rbd (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:01.627 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up rbd-fuse (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:01.753 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-rgw (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:01.879 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up python3-cephfs (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:01.988 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-common (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:02.490 INFO:teuthology.orchestra.run.smithi183.stdout:Adding group ceph....done 2023-10-13T23:59:02.849 INFO:teuthology.orchestra.run.smithi183.stdout:Adding system user ceph....done 2023-10-13T23:59:03.026 INFO:teuthology.orchestra.run.smithi183.stdout:Setting system user ceph properties....done 2023-10-13T23:59:03.040 INFO:teuthology.orchestra.run.smithi183.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-13T23:59:03.186 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-13T23:59:03.523 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-13T23:59:04.014 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-test (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:04.116 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up radosgw (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:04.581 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-13T23:59:04.582 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-13T23:59:05.084 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-base (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:05.436 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-13T23:59:05.969 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mds (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:06.216 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-13T23:59:06.217 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-13T23:59:06.788 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:06.990 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-13T23:59:06.991 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-13T23:59:07.531 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-osd (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:07.861 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-13T23:59:07.861 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-13T23:59:08.367 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:08.492 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:08.644 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mon (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:08.859 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-13T23:59:08.859 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-13T23:59:09.378 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-cephadm (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:09.529 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:09.655 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-mgr-dashboard (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:09.806 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up ceph-volume (18.0.0-6724-g01fff6a7-1jammy) ... 2023-10-13T23:59:10.251 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-13T23:59:10.959 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-13T23:59:11.425 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-13T23:59:11.428 DEBUG:teuthology.parallel:result is None 2023-10-13T23:59:11.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=01fff6a72a328459c1d153e5dc1de6a34e48a82f 2023-10-13T23:59:11.648 DEBUG:teuthology.orchestra.run.smithi183:> dpkg-query -W -f '${Version}' ceph 2023-10-13T23:59:11.668 INFO:teuthology.orchestra.run.smithi183.stdout:18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:59:11.668 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:59:11.669 INFO:teuthology.task.install:The correct ceph version 18.0.0-6724-g01fff6a7-1jammy is installed. 2023-10-13T23:59:11.669 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-13T23:59:11.670 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:11.670 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-13T23:59:11.723 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-13T23:59:11.723 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:11.724 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2023-10-13T23:59:11.778 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-13T23:59:11.836 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-13T23:59:11.836 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:11.837 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-13T23:59:11.889 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-13T23:59:11.941 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-13T23:59:11.942 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:11.942 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2023-10-13T23:59:11.997 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-13T23:59:12.050 INFO:teuthology.run_tasks:Running task ceph... 2023-10-13T23:59:12.118 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-13T23:59:12.119 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2023-10-13T23:59:12.133 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-13T23:59:12.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-13T23:59:12.185 INFO:tasks.ceph:Creating extra log directories... 2023-10-13T23:59:12.186 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-13T23:59:12.250 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-13T23:59:12.251 INFO:tasks.ceph:config {'conf': {'client': {'rbd default features': 125}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'debug rbd': 20}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-13T23:59:12.251 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259', 'branch': 'wip-dis-testing', 'description': 'rbd/cli/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} data-pool/none features/journaling msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd_support_module_recovery}', 'email': 'idryomov@gmail.com', 'first_in_suite': False, 'job_id': '7427259', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-dis-testing'}, 'ceph': {'conf': {'client': {'rbd default features': 125}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1, 'debug rbd': 20}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'sha1': '01fff6a72a328459c1d153e5dc1de6a34e48a82f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '01fff6a72a328459c1d153e5dc1de6a34e48a82f'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-dis-testing-qa', 'sha1': '735f68d7bf91d643813a8420fc2b11441f4bd8c4'}}, 'owner': 'scheduled_dis@teuthology', 'priority': 74, 'repo': 'https://github.com/ceph/ceph-ci', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 2449, 'sha1': '01fff6a72a328459c1d153e5dc1de6a34e48a82f', 'sleep_before_teardown': 0, 'subset': '49/99', 'suite': 'rbd', 'suite_branch': 'wip-dis-testing-qa', 'suite_path': '/home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/idryomov/ceph', 'suite_sha1': '735f68d7bf91d643813a8420fc2b11441f4bd8c4', 'targets': {'smithi183.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH/vh7O2KCB/0ptwMSzzgY+G3dboygsXf4i9tzvtMR/JZ7YzaIu2HcgDfLzHJ7Tqa+552PeXue3WWel93DCIE20='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'install': {'extra_system_packages': ['fio']}}, {'workunit': {'clients': {'client.0': ['rbd/rbd_support_module_recovery.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-13_23:35:38', 'tube': 'smithi', 'user': 'dis', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-13T23:59:12.252 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-13T23:59:12.291 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2023-10-13T23:59:12.346 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:12.346 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2023-10-13T23:59:12.395 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-13T23:59:12.395 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2023-10-13T23:59:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-13T23:59:12.443 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-13T23:59:12.443 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 886 Links: 1 2023-10-13T23:59:12.444 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-13T23:59:12.444 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-13 23:54:07.160786148 +0000 2023-10-13T23:59:12.445 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-13 23:54:06.800789364 +0000 2023-10-13T23:59:12.445 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-13 23:54:06.800789364 +0000 2023-10-13T23:59:12.445 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-10-13T23:59:12.446 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-13T23:59:12.497 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-13T23:59:12.498 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-13T23:59:12.498 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000455497 s, 1.1 MB/s 2023-10-13T23:59:12.499 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-13T23:59:12.549 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2023-10-13T23:59:12.594 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-13T23:59:12.595 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-13T23:59:12.595 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 905 Links: 1 2023-10-13T23:59:12.595 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-13T23:59:12.595 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-13 23:54:07.744780933 +0000 2023-10-13T23:59:12.596 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-13 23:54:07.384784148 +0000 2023-10-13T23:59:12.596 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-13 23:54:07.384784148 +0000 2023-10-13T23:59:12.596 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-10-13T23:59:12.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-13T23:59:12.649 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-13T23:59:12.649 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-13T23:59:12.650 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000299554 s, 1.7 MB/s 2023-10-13T23:59:12.651 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-13T23:59:12.700 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2023-10-13T23:59:12.746 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-13T23:59:12.747 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-13T23:59:12.747 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 920 Links: 1 2023-10-13T23:59:12.747 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-13T23:59:12.747 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-13 23:54:08.332775679 +0000 2023-10-13T23:59:12.748 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-13 23:54:08.000778645 +0000 2023-10-13T23:59:12.748 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-13 23:54:08.000778645 +0000 2023-10-13T23:59:12.748 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-10-13T23:59:12.748 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-13T23:59:12.801 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-13T23:59:12.801 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-13T23:59:12.802 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000441847 s, 1.2 MB/s 2023-10-13T23:59:12.803 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-13T23:59:12.854 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2023-10-13T23:59:12.903 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-13T23:59:12.903 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-13T23:59:12.903 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 940 Links: 1 2023-10-13T23:59:12.903 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-13T23:59:12.904 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2023-10-13 23:54:08.896770643 +0000 2023-10-13T23:59:12.904 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2023-10-13 23:54:08.560773644 +0000 2023-10-13T23:59:12.904 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2023-10-13 23:54:08.560773644 +0000 2023-10-13T23:59:12.904 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: - 2023-10-13T23:59:12.904 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-13T23:59:12.957 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2023-10-13T23:59:12.958 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2023-10-13T23:59:12.958 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000391851 s, 1.3 MB/s 2023-10-13T23:59:12.959 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-13T23:59:13.007 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-13T23:59:13.008 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-10-13T23:59:13.008 INFO:tasks.ceph:Generating config... 2023-10-13T23:59:13.009 INFO:tasks.ceph:[client] rbd default features = 125 2023-10-13T23:59:13.010 INFO:tasks.ceph:[global] mon client directed command retry = 5 2023-10-13T23:59:13.010 INFO:tasks.ceph:[global] mon warn on pool no app = False 2023-10-13T23:59:13.010 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2023-10-13T23:59:13.010 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-13T23:59:13.011 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-13T23:59:13.011 INFO:tasks.ceph:[mgr] debug rbd = 20 2023-10-13T23:59:13.011 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-13T23:59:13.011 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-13T23:59:13.012 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-13T23:59:13.012 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-13T23:59:13.012 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2023-10-13T23:59:13.013 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-10-13T23:59:13.013 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-13T23:59:13.013 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-13T23:59:13.013 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-13T23:59:13.013 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-13T23:59:13.014 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-13T23:59:13.014 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-13T23:59:13.014 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-13T23:59:13.015 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-13T23:59:13.015 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-13T23:59:13.015 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-13T23:59:13.015 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-13T23:59:13.016 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2023-10-13T23:59:13.016 INFO:tasks.ceph:Setting up mon.a... 2023-10-13T23:59:13.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-13T23:59:13.080 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.keyring 2023-10-13T23:59:13.089 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-13T23:59:13.162 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-13T23:59:13.218 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.183')] 2023-10-13T23:59:13.219 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.183', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug rbd': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd default features': 125}, 'mon.a': {}} 2023-10-13T23:59:13.219 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:13.220 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-13T23:59:13.266 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.183 --print /home/ubuntu/cephtest/ceph.monmap 2023-10-13T23:59:13.344 INFO:teuthology.orchestra.run.smithi183.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-13T23:59:13.344 INFO:teuthology.orchestra.run.smithi183.stdout:monmaptool: generated fsid 01fe18cb-485e-439b-9438-7a52435d33f2 2023-10-13T23:59:13.345 INFO:teuthology.orchestra.run.smithi183.stdout:setting min_mon_release = pacific 2023-10-13T23:59:13.345 INFO:teuthology.orchestra.run.smithi183.stdout:epoch 0 2023-10-13T23:59:13.345 INFO:teuthology.orchestra.run.smithi183.stdout:fsid 01fe18cb-485e-439b-9438-7a52435d33f2 2023-10-13T23:59:13.345 INFO:teuthology.orchestra.run.smithi183.stdout:last_changed 2023-10-13T23:59:13.346571+0000 2023-10-13T23:59:13.346 INFO:teuthology.orchestra.run.smithi183.stdout:created 2023-10-13T23:59:13.346571+0000 2023-10-13T23:59:13.346 INFO:teuthology.orchestra.run.smithi183.stdout:min_mon_release 16 (pacific) 2023-10-13T23:59:13.346 INFO:teuthology.orchestra.run.smithi183.stdout:election_strategy: 1 2023-10-13T23:59:13.346 INFO:teuthology.orchestra.run.smithi183.stdout:0: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.a 2023-10-13T23:59:13.346 INFO:teuthology.orchestra.run.smithi183.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-10-13T23:59:13.348 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-13T23:59:13.395 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 01fe18cb-485e-439b-9438-7a52435d33f2... 2023-10-13T23:59:13.396 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-13T23:59:13.467 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2023-10-13T23:59:13.467 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2023-10-13T23:59:13.467 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-13T23:59:13.468 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2023-10-13T23:59:13.468 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.468 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2023-10-13T23:59:13.468 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.469 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2023-10-13T23:59:13.469 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.469 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2023-10-13T23:59:13.469 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2023-10-13T23:59:13.469 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.469 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2023-10-13T23:59:13.470 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2023-10-13T23:59:13.470 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.470 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-13T23:59:13.470 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2023-10-13T23:59:13.470 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.471 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2023-10-13T23:59:13.471 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-13T23:59:13.471 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.471 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2023-10-13T23:59:13.471 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.473 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2023-10-13T23:59:13.473 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2023-10-13T23:59:13.473 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2023-10-13T23:59:13.474 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2023-10-13T23:59:13.474 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2023-10-13T23:59:13.474 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2023-10-13T23:59:13.474 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2023-10-13T23:59:13.474 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-13T23:59:13.475 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-13T23:59:13.475 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2023-10-13T23:59:13.475 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.475 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-13T23:59:13.476 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.476 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2023-10-13T23:59:13.476 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.476 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2023-10-13T23:59:13.476 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.477 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2023-10-13T23:59:13.477 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2023-10-13T23:59:13.477 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2023-10-13T23:59:13.477 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = 172.21.15.183 2023-10-13T23:59:13.477 INFO:teuthology.orchestra.run.smithi183.stdout: mon client directed command retry = 5 2023-10-13T23:59:13.478 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on pool no app = False 2023-10-13T23:59:13.478 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject socket failures = 5000 2023-10-13T23:59:13.478 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = 01fe18cb-485e-439b-9438-7a52435d33f2 2023-10-13T23:59:13.478 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.478 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2023-10-13T23:59:13.479 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2023-10-13T23:59:13.479 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.479 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2023-10-13T23:59:13.479 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2023-10-13T23:59:13.479 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock profile = high_recovery_ops 2023-10-13T23:59:13.479 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.480 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2023-10-13T23:59:13.480 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2023-10-13T23:59:13.480 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.480 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2023-10-13T23:59:13.480 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2023-10-13T23:59:13.481 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2023-10-13T23:59:13.481 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.481 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2023-10-13T23:59:13.481 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2023-10-13T23:59:13.482 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.482 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2023-10-13T23:59:13.482 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.482 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2023-10-13T23:59:13.482 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.483 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2023-10-13T23:59:13.483 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.483 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2023-10-13T23:59:13.483 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2023-10-13T23:59:13.483 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.484 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2023-10-13T23:59:13.484 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2023-10-13T23:59:13.484 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2023-10-13T23:59:13.484 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression algorithm = lz4 2023-10-13T23:59:13.484 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore compression mode = aggressive 2023-10-13T23:59:13.485 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2023-10-13T23:59:13.485 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 1/20 2023-10-13T23:59:13.485 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 1/20 2023-10-13T23:59:13.485 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-13T23:59:13.485 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 20 2023-10-13T23:59:13.485 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 4/10 2023-10-13T23:59:13.486 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2023-10-13T23:59:13.486 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2023-10-13T23:59:13.486 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2023-10-13T23:59:13.486 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2023-10-13T23:59:13.487 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2023-10-13T23:59:13.487 INFO:teuthology.orchestra.run.smithi183.stdout: osd shutdown pgref assert = True 2023-10-13T23:59:13.487 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.487 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2023-10-13T23:59:13.487 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-13T23:59:13.488 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2023-10-13T23:59:13.488 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2023-10-13T23:59:13.488 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2023-10-13T23:59:13.488 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2023-10-13T23:59:13.488 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2023-10-13T23:59:13.489 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2023-10-13T23:59:13.489 INFO:teuthology.orchestra.run.smithi183.stdout: debug rbd = 20 2023-10-13T23:59:13.489 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.489 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2023-10-13T23:59:13.489 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2023-10-13T23:59:13.490 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2023-10-13T23:59:13.490 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2023-10-13T23:59:13.490 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2023-10-13T23:59:13.490 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2023-10-13T23:59:13.490 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2023-10-13T23:59:13.491 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2023-10-13T23:59:13.491 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2023-10-13T23:59:13.491 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2023-10-13T23:59:13.491 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2023-10-13T23:59:13.491 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.491 INFO:teuthology.orchestra.run.smithi183.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-13T23:59:13.492 INFO:teuthology.orchestra.run.smithi183.stdout: auth mon ticket ttl = 660 # 11m 2023-10-13T23:59:13.492 INFO:teuthology.orchestra.run.smithi183.stdout: auth service ticket ttl = 240 # 4m 2023-10-13T23:59:13.492 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.492 INFO:teuthology.orchestra.run.smithi183.stdout: # don't complain about insecure global_id in the test suite 2023-10-13T23:59:13.492 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-13T23:59:13.493 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-13T23:59:13.493 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.493 INFO:teuthology.orchestra.run.smithi183.stdout: # 1m isn't quite enough 2023-10-13T23:59:13.493 INFO:teuthology.orchestra.run.smithi183.stdout: mon_down_mkfs_grace = 2m 2023-10-13T23:59:13.493 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.494 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_filestore_osds = false 2023-10-13T23:59:13.494 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:13.494 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2023-10-13T23:59:13.494 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2023-10-13T23:59:13.494 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2023-10-13T23:59:13.495 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2023-10-13T23:59:13.495 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-13T23:59:13.495 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-13T23:59:13.495 INFO:teuthology.orchestra.run.smithi183.stdout: rbd default features = 125 2023-10-13T23:59:13.495 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2023-10-13T23:59:13.496 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-13T23:59:13.497 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-13T23:59:13.587 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-13T23:59:13.587 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:13.587 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-13T23:59:13.592 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:13.592 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-13T23:59:13.638 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2023-10-13T23:59:13.638 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:13.638 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-13T23:59:13.639 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-13T23:59:13.702 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:13.702 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-13T23:59:13.750 INFO:tasks.ceph:Setting up mon nodes... 2023-10-13T23:59:13.751 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-13T23:59:13.751 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-13T23:59:13.835 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-13T23:59:13.838 INFO:tasks.ceph:Setting up mds nodes... 2023-10-13T23:59:13.838 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-13T23:59:13.839 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-13T23:59:13.913 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-13T23:59:13.923 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-13T23:59:13.924 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2023-10-13T23:59:13.924 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-13T23:59:13.976 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-13T23:59:13.976 INFO:tasks.ceph:role: osd.0 2023-10-13T23:59:13.977 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-13T23:59:13.977 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-13T23:59:14.038 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-13T23:59:14.038 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-13T23:59:14.039 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-13T23:59:14.039 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-13T23:59:14.039 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-13T23:59:14.039 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-10-13T23:59:14.039 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-13T23:59:14.040 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-13T23:59:14.040 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-13T23:59:14.040 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-13T23:59:14.504 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-10-13T23:59:14.506 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-10-13T23:59:14.506 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-13T23:59:14.596 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-13T23:59:14.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-13T23:59:14.608 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2023-10-13T23:59:14.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-13T23:59:14.661 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-13T23:59:14.661 INFO:tasks.ceph:role: osd.1 2023-10-13T23:59:14.662 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-13T23:59:14.662 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-13T23:59:14.723 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-13T23:59:14.724 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-13T23:59:14.724 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-13T23:59:14.724 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-13T23:59:14.724 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-13T23:59:14.724 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-10-13T23:59:14.725 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-13T23:59:14.725 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-13T23:59:14.725 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-13T23:59:14.725 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-13T23:59:15.189 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-10-13T23:59:15.192 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-10-13T23:59:15.192 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-13T23:59:15.338 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-13T23:59:15.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-13T23:59:15.351 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2023-10-13T23:59:15.352 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-13T23:59:15.405 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2023-10-13T23:59:15.406 INFO:tasks.ceph:role: osd.2 2023-10-13T23:59:15.406 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-13T23:59:15.406 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-13T23:59:15.474 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-13T23:59:15.474 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-13T23:59:15.474 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-13T23:59:15.475 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-13T23:59:15.475 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-13T23:59:15.475 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2023-10-13T23:59:15.475 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-13T23:59:15.476 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-13T23:59:15.476 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-13T23:59:15.476 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-13T23:59:15.950 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2023-10-13T23:59:15.952 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2023-10-13T23:59:15.953 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-13T23:59:16.087 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-13T23:59:16.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-13T23:59:16.099 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: /sbin/restorecon: command not found 2023-10-13T23:59:16.099 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-13T23:59:16.200 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:16.196+0000 7f94bd3f4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-13T23:59:16.204 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:16.200+0000 7f94bd3f4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-13T23:59:16.205 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:16.200+0000 7f94bd3f4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-13T23:59:16.205 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:16.200+0000 7f94bd3f4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-13T23:59:16.205 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:16.200+0000 7f94bd3f4640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-13T23:59:16.206 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:16.200+0000 7f94bd3f4640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-13T23:59:19.243 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-13T23:59:19.258 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-13T23:59:19.341 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:19.336+0000 7f28943ac640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-13T23:59:19.342 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:19.336+0000 7f28943ac640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-13T23:59:19.342 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:19.336+0000 7f28943ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-13T23:59:19.342 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:19.336+0000 7f28943ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-13T23:59:19.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:19.340+0000 7f28943ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-13T23:59:19.343 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:19.340+0000 7f28943ac640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-13T23:59:22.345 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-13T23:59:22.399 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-13T23:59:22.481 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:22.476+0000 7f40b9196640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-13T23:59:22.482 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:22.476+0000 7f40b9196640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-13T23:59:22.482 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:22.476+0000 7f40b9196640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-13T23:59:22.483 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:22.480+0000 7f40b9196640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-13T23:59:22.485 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:22.480+0000 7f40b9196640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-13T23:59:22.485 INFO:teuthology.orchestra.run.smithi183.stderr:2023-10-13T23:59:22.480+0000 7f40b9196640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-13T23:59:25.497 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-13T23:59:25.554 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-13T23:59:25.555 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:25.555 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-13T23:59:25.609 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:25.609 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-13T23:59:25.661 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:25.661 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-13T23:59:25.713 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:25.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-13T23:59:25.770 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:25.771 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-13T23:59:25.818 INFO:tasks.ceph:Adding keys to all mons... 2023-10-13T23:59:25.819 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-13T23:59:25.872 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2023-10-13T23:59:25.872 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDR2SllYyzsMRAAbDeC/iWEaim5sVSgXsJnoQ== 2023-10-13T23:59:25.873 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2023-10-13T23:59:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDU2Sll7XMSDBAAFucGN81SkVSma2irZMkSuQ== 2023-10-13T23:59:25.873 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2023-10-13T23:59:25.873 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDX2SllGtR5FBAAQoVwqRf9tUog4Ixl6z3ZLA== 2023-10-13T23:59:25.873 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2023-10-13T23:59:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDa2Sll08bRHBAAA50c/2WbLMAomakUX0KvcQ== 2023-10-13T23:59:25.874 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2023-10-13T23:59:25.874 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQDR2SllZleXNhAABeX9wTtUoa4MgMKiX4HO/A== 2023-10-13T23:59:25.875 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-13T23:59:25.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-13T23:59:26.006 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-13T23:59:26.088 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-13T23:59:26.167 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-13T23:59:26.255 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-13T23:59:26.255 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-13T23:59:26.266 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-13T23:59:26.363 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-13T23:59:26.374 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-13T23:59:26.421 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-13T23:59:26.422 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-13T23:59:26.422 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-13T23:59:26.463 INFO:tasks.ceph.mon.a:Started 2023-10-13T23:59:26.464 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-13T23:59:26.464 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-13T23:59:26.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-13T23:59:26.466 INFO:tasks.ceph.mgr.x:Started 2023-10-13T23:59:26.467 INFO:tasks.ceph:Setting crush tunables to default 2023-10-13T23:59:26.467 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd crush tunables default 2023-10-13T23:59:26.718 INFO:teuthology.orchestra.run.smithi183.stderr:adjusted tunables profile to default 2023-10-13T23:59:26.736 INFO:tasks.ceph:check_enable_crimson: False 2023-10-13T23:59:26.736 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-13T23:59:26.737 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:26.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-13T23:59:26.749 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:26.750 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-13T23:59:26.809 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:26.809 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-13T23:59:26.863 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:26.860+0000 7f6674297180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-13T23:59:26.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new c0949fc6-f124-483d-bc0d-21287eb4d027 0 2023-10-13T23:59:27.100 INFO:teuthology.orchestra.run.smithi183.stdout:0 2023-10-13T23:59:27.118 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new defa036c-6c18-43a4-87d4-a66a3cb829bd 1 2023-10-13T23:59:27.311 INFO:teuthology.orchestra.run.smithi183.stdout:1 2023-10-13T23:59:27.329 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new db6b0fb5-f6a4-4e74-b34a-4cb459386346 2 2023-10-13T23:59:27.417 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:27.412+0000 7f6674297180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-13T23:59:27.516 INFO:teuthology.orchestra.run.smithi183.stdout:2 2023-10-13T23:59:27.533 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-13T23:59:27.533 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-13T23:59:27.536 INFO:tasks.ceph.osd.0:Started 2023-10-13T23:59:27.536 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-13T23:59:27.537 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-13T23:59:27.539 INFO:tasks.ceph.osd.1:Started 2023-10-13T23:59:27.539 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-13T23:59:27.539 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-13T23:59:27.542 INFO:tasks.ceph.osd.2:Started 2023-10-13T23:59:27.542 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-13T23:59:27.622 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:27.620+0000 7f6674297180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-13T23:59:27.717 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:27.717 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":5,"fsid":"01fe18cb-485e-439b-9438-7a52435d33f2","created":"2023-10-13T23:59:26.603673+0000","modified":"2023-10-13T23:59:27.517541+0000","last_up_change":"0.000000","last_in_change":"2023-10-13T23:59:27.517541+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c0949fc6-f124-483d-bc0d-21287eb4d027","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"defa036c-6c18-43a4-87d4-a66a3cb829bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"db6b0fb5-f6a4-4e74-b34a-4cb459386346","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-13T23:59:27.733 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-13T23:59:27.733 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-13T23:59:28.035 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-13T23:59:28.166 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-13T23:59:28.164+0000 7f1ded2e8640 -1 Falling back to public interface 2023-10-13T23:59:28.174 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-13T23:59:28.172+0000 7ff192304640 -1 Falling back to public interface 2023-10-13T23:59:28.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-13T23:59:28.172+0000 7faefd82a640 -1 Falling back to public interface 2023-10-13T23:59:28.201 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.196+0000 7f6674297180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-13T23:59:28.210 INFO:teuthology.misc.health.smithi183.stdout: 2023-10-13T23:59:28.210 INFO:teuthology.misc.health.smithi183.stdout:{"epoch":5,"fsid":"01fe18cb-485e-439b-9438-7a52435d33f2","created":"2023-10-13T23:59:26.603673+0000","modified":"2023-10-13T23:59:27.517541+0000","last_up_change":"0.000000","last_in_change":"2023-10-13T23:59:27.517541+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c0949fc6-f124-483d-bc0d-21287eb4d027","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"defa036c-6c18-43a4-87d4-a66a3cb829bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"db6b0fb5-f6a4-4e74-b34a-4cb459386346","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-13T23:59:28.225 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-10-13T23:59:28.257 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.252+0000 7f6674297180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-13T23:59:28.309 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.304+0000 7f6674297180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-13T23:59:28.385 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.380+0000 7f6674297180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-13T23:59:28.436 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.432+0000 7f6674297180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-13T23:59:28.513 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.508+0000 7f6674297180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-13T23:59:28.565 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.560+0000 7f6674297180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-13T23:59:28.704 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.700+0000 7f6674297180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-13T23:59:28.763 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:28.760+0000 7f6674297180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-13T23:59:29.401 INFO:tasks.ceph.mgr.x.smithi183.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-13T23:59:29.401 INFO:tasks.ceph.mgr.x.smithi183.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-13T23:59:29.401 INFO:tasks.ceph.mgr.x.smithi183.stderr: from numpy import show_config as show_numpy_config 2023-10-13T23:59:29.431 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:29.428+0000 7f6674297180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-13T23:59:29.554 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:29.552+0000 7f6674297180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-13T23:59:29.712 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:29.708+0000 7f6674297180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-13T23:59:29.775 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:29.772+0000 7f6674297180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-13T23:59:29.897 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:29.892+0000 7f6674297180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-13T23:59:30.509 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:30.504+0000 7f6674297180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-13T23:59:30.584 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:30.580+0000 7f6674297180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-13T23:59:30.753 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:30.748+0000 7f6674297180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-13T23:59:30.827 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:30.824+0000 7f6674297180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-13T23:59:31.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:31.124+0000 7f6674297180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-13T23:59:31.187 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:31.184+0000 7f6674297180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-13T23:59:32.019 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-13T23:59:32.016+0000 7f1ded2e8640 -1 osd.0 0 log_to_monitors true 2023-10-13T23:59:32.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-13T23:59:32.024+0000 7faefd82a640 -1 osd.1 0 log_to_monitors true 2023-10-13T23:59:32.073 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-13T23:59:32.068+0000 7ff192304640 -1 osd.2 0 log_to_monitors true 2023-10-13T23:59:32.231 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-13T23:59:32.228+0000 7f661ba0a640 -1 mgr.server handle_report got status from non-daemon mon.a 2023-10-13T23:59:34.528 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-13T23:59:34.718 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-13T23:59:34.716+0000 7ff18e2ac640 -1 osd.2 0 waiting for initial osdmap 2023-10-13T23:59:34.721 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-13T23:59:34.716+0000 7f1de9290640 -1 osd.0 0 waiting for initial osdmap 2023-10-13T23:59:34.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-13T23:59:34.724+0000 7faef97d2640 -1 osd.1 0 waiting for initial osdmap 2023-10-13T23:59:34.762 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-13T23:59:34.760+0000 7f1de10c5640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-13T23:59:34.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-13T23:59:34.760+0000 7faef1607640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-13T23:59:34.763 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-13T23:59:34.760+0000 7ff1860e1640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-13T23:59:34.857 INFO:teuthology.misc.health.smithi183.stdout: 2023-10-13T23:59:34.857 INFO:teuthology.misc.health.smithi183.stdout:{"epoch":7,"fsid":"01fe18cb-485e-439b-9438-7a52435d33f2","created":"2023-10-13T23:59:26.603673+0000","modified":"2023-10-13T23:59:33.549500+0000","last_up_change":"0.000000","last_in_change":"2023-10-13T23:59:27.517541+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c0949fc6-f124-483d-bc0d-21287eb4d027","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"defa036c-6c18-43a4-87d4-a66a3cb829bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"db6b0fb5-f6a4-4e74-b34a-4cb459386346","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-13T23:59:34.873 DEBUG:teuthology.misc:0 of 3 OSDs are up 2023-10-13T23:59:41.176 DEBUG:teuthology.orchestra.run.smithi183:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-13T23:59:41.466 INFO:teuthology.misc.health.smithi183.stdout: 2023-10-13T23:59:41.466 INFO:teuthology.misc.health.smithi183.stdout:{"epoch":12,"fsid":"01fe18cb-485e-439b-9438-7a52435d33f2","created":"2023-10-13T23:59:26.603673+0000","modified":"2023-10-13T23:59:39.759231+0000","last_up_change":"2023-10-13T23:59:35.733023+0000","last_in_change":"2023-10-13T23:59:27.517541+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-13T23:59:37.286490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c0949fc6-f124-483d-bc0d-21287eb4d027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6801","nonce":973214594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6803","nonce":973214594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6807","nonce":973214594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6805","nonce":973214594}]},"public_addr":"172.21.15.183:6801/973214594","cluster_addr":"172.21.15.183:6803/973214594","heartbeat_back_addr":"172.21.15.183:6807/973214594","heartbeat_front_addr":"172.21.15.183:6805/973214594","state":["exists","up"]},{"osd":1,"uuid":"defa036c-6c18-43a4-87d4-a66a3cb829bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6811","nonce":3579441550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6815","nonce":3579441550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6821","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6823","nonce":3579441550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6819","nonce":3579441550}]},"public_addr":"172.21.15.183:6811/3579441550","cluster_addr":"172.21.15.183:6815/3579441550","heartbeat_back_addr":"172.21.15.183:6823/3579441550","heartbeat_front_addr":"172.21.15.183:6819/3579441550","state":["exists","up"]},{"osd":2,"uuid":"db6b0fb5-f6a4-4e74-b34a-4cb459386346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6810","nonce":869421319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6814","nonce":869421319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6822","nonce":869421319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6818","nonce":869421319}]},"public_addr":"172.21.15.183:6810/869421319","cluster_addr":"172.21.15.183:6814/869421319","heartbeat_back_addr":"172.21.15.183:6822/869421319","heartbeat_front_addr":"172.21.15.183:6818/869421319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:32.985796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.024705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.041682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-13T23:59:41.483 DEBUG:teuthology.misc:3 of 3 OSDs are up 2023-10-13T23:59:41.484 INFO:tasks.ceph:Creating RBD pool 2023-10-13T23:59:41.484 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-13T23:59:42.780 INFO:teuthology.orchestra.run.smithi183.stderr:pool 'rbd' created 2023-10-13T23:59:42.808 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-13T23:59:43.786 INFO:teuthology.orchestra.run.smithi183.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-13T23:59:43.820 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-13T23:59:43.821 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-13T23:59:43.821 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-13T23:59:43.821 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-13T23:59:43.822 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-13T23:59:44.156 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:44.174 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6824","nonce":4075389088},{"type":"v1","addr":"172.21.15.183:6825","nonce":4075389088}]},"active_addr":"172.21.15.183:6825/4075389088","active_change":"2023-10-13T23:59:31.209288+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":3021386618}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":580480178}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":2449655932}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.183:0","nonce":4289861151}]}]} 2023-10-13T23:59:44.176 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-13T23:59:44.177 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-13T23:59:44.177 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-13T23:59:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:44.460 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":14,"fsid":"01fe18cb-485e-439b-9438-7a52435d33f2","created":"2023-10-13T23:59:26.603673+0000","modified":"2023-10-13T23:59:43.782877+0000","last_up_change":"2023-10-13T23:59:35.733023+0000","last_in_change":"2023-10-13T23:59:27.517541+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-13T23:59:37.286490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-13T23:59:41.813626+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c0949fc6-f124-483d-bc0d-21287eb4d027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6801","nonce":973214594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6803","nonce":973214594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6807","nonce":973214594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6805","nonce":973214594}]},"public_addr":"172.21.15.183:6801/973214594","cluster_addr":"172.21.15.183:6803/973214594","heartbeat_back_addr":"172.21.15.183:6807/973214594","heartbeat_front_addr":"172.21.15.183:6805/973214594","state":["exists","up"]},{"osd":1,"uuid":"defa036c-6c18-43a4-87d4-a66a3cb829bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6811","nonce":3579441550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6815","nonce":3579441550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6821","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6823","nonce":3579441550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6819","nonce":3579441550}]},"public_addr":"172.21.15.183:6811/3579441550","cluster_addr":"172.21.15.183:6815/3579441550","heartbeat_back_addr":"172.21.15.183:6823/3579441550","heartbeat_front_addr":"172.21.15.183:6819/3579441550","state":["exists","up"]},{"osd":2,"uuid":"db6b0fb5-f6a4-4e74-b34a-4cb459386346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6810","nonce":869421319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6814","nonce":869421319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6822","nonce":869421319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6818","nonce":869421319}]},"public_addr":"172.21.15.183:6810/869421319","cluster_addr":"172.21.15.183:6814/869421319","heartbeat_back_addr":"172.21.15.183:6822/869421319","heartbeat_front_addr":"172.21.15.183:6818/869421319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:32.985796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.024705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.041682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-13T23:59:44.477 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-13T23:59:44.478 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-13T23:59:44.772 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:44.773 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":14,"fsid":"01fe18cb-485e-439b-9438-7a52435d33f2","created":"2023-10-13T23:59:26.603673+0000","modified":"2023-10-13T23:59:43.782877+0000","last_up_change":"2023-10-13T23:59:35.733023+0000","last_in_change":"2023-10-13T23:59:27.517541+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-13T23:59:37.286490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-13T23:59:41.813626+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c0949fc6-f124-483d-bc0d-21287eb4d027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6801","nonce":973214594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6803","nonce":973214594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6807","nonce":973214594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6805","nonce":973214594}]},"public_addr":"172.21.15.183:6801/973214594","cluster_addr":"172.21.15.183:6803/973214594","heartbeat_back_addr":"172.21.15.183:6807/973214594","heartbeat_front_addr":"172.21.15.183:6805/973214594","state":["exists","up"]},{"osd":1,"uuid":"defa036c-6c18-43a4-87d4-a66a3cb829bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6811","nonce":3579441550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6815","nonce":3579441550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6821","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6823","nonce":3579441550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6819","nonce":3579441550}]},"public_addr":"172.21.15.183:6811/3579441550","cluster_addr":"172.21.15.183:6815/3579441550","heartbeat_back_addr":"172.21.15.183:6823/3579441550","heartbeat_front_addr":"172.21.15.183:6819/3579441550","state":["exists","up"]},{"osd":2,"uuid":"db6b0fb5-f6a4-4e74-b34a-4cb459386346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6810","nonce":869421319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6814","nonce":869421319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6822","nonce":869421319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6818","nonce":869421319}]},"public_addr":"172.21.15.183:6810/869421319","cluster_addr":"172.21.15.183:6814/869421319","heartbeat_back_addr":"172.21.15.183:6822/869421319","heartbeat_front_addr":"172.21.15.183:6818/869421319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:32.985796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.024705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.041682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-13T23:59:44.791 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-13T23:59:44.791 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-13T23:59:44.792 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-13T23:59:44.944 INFO:teuthology.orchestra.run.smithi183.stdout:34359738372 2023-10-13T23:59:44.945 INFO:teuthology.orchestra.run.smithi183.stdout:34359738373 2023-10-13T23:59:44.946 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-13T23:59:44.946 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-13T23:59:44.951 INFO:teuthology.orchestra.run.smithi183.stdout:34359738372 2023-10-13T23:59:44.951 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-13T23:59:45.267 INFO:teuthology.orchestra.run.smithi183.stdout:34359738371 2023-10-13T23:59:45.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.2 2023-10-13T23:59:45.292 INFO:teuthology.orchestra.run.smithi183.stdout:34359738372 2023-10-13T23:59:45.295 INFO:teuthology.orchestra.run.smithi183.stdout:34359738371 2023-10-13T23:59:45.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.1 2023-10-13T23:59:45.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2023-10-13T23:59:46.285 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-13T23:59:46.312 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-13T23:59:46.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-13T23:59:46.589 INFO:teuthology.orchestra.run.smithi183.stdout:34359738372 2023-10-13T23:59:46.607 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.2 2023-10-13T23:59:46.607 DEBUG:teuthology.parallel:result is None 2023-10-13T23:59:46.615 INFO:teuthology.orchestra.run.smithi183.stdout:34359738373 2023-10-13T23:59:46.621 INFO:teuthology.orchestra.run.smithi183.stdout:34359738372 2023-10-13T23:59:46.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.1 2023-10-13T23:59:46.634 DEBUG:teuthology.parallel:result is None 2023-10-13T23:59:46.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.0 2023-10-13T23:59:46.640 DEBUG:teuthology.parallel:result is None 2023-10-13T23:59:46.640 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-13T23:59:46.640 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-13T23:59:46.945 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:46.946 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-13T23:59:46.961 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-10-13T23:59:45.217758+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81532,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283033988,"statfs":{"total":289910292480,"available":289826803712,"internally_reserved":0,"allocated":483328,"data_stored":1281896,"data_compressed":12514,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4782,"internal_metadata":82963794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.434085"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800713+0000","last_change":"2023-10-13T23:59:43.806959+0000","last_active":"2023-10-13T23:59:44.800713+0000","last_peered":"2023-10-13T23:59:44.800713+0000","last_clean":"2023-10-13T23:59:44.800713+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-13T23:59:44.800713+0000","last_undegraded":"2023-10-13T23:59:44.800713+0000","last_fullsized":"2023-10-13T23:59:44.800713+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:45:57.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800643+0000","last_change":"2023-10-13T23:59:43.807975+0000","last_active":"2023-10-13T23:59:44.800643+0000","last_peered":"2023-10-13T23:59:44.800643+0000","last_clean":"2023-10-13T23:59:44.800643+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-13T23:59:44.800643+0000","last_undegraded":"2023-10-13T23:59:44.800643+0000","last_fullsized":"2023-10-13T23:59:44.800643+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:54:08.157126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800689+0000","last_change":"2023-10-13T23:59:43.808584+0000","last_active":"2023-10-13T23:59:44.800689+0000","last_peered":"2023-10-13T23:59:44.800689+0000","last_clean":"2023-10-13T23:59:44.800689+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-13T23:59:44.800689+0000","last_undegraded":"2023-10-13T23:59:44.800689+0000","last_fullsized":"2023-10-13T23:59:44.800689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T05:22:55.056177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800562+0000","last_change":"2023-10-13T23:59:43.807900+0000","last_active":"2023-10-13T23:59:44.800562+0000","last_peered":"2023-10-13T23:59:44.800562+0000","last_clean":"2023-10-13T23:59:44.800562+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-13T23:59:44.800562+0000","last_undegraded":"2023-10-13T23:59:44.800562+0000","last_fullsized":"2023-10-13T23:59:44.800562+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:14:06.611855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-13T23:59:43.811983+0000","last_change":"2023-10-13T23:59:43.806111+0000","last_active":"2023-10-13T23:59:43.811983+0000","last_peered":"2023-10-13T23:59:43.811983+0000","last_clean":"2023-10-13T23:59:43.811983+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-13T23:59:43.811983+0000","last_undegraded":"2023-10-13T23:59:43.811983+0000","last_fullsized":"2023-10-13T23:59:43.811983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:14:23.959646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-13T23:59:43.806655+0000","last_change":"2023-10-13T23:59:43.803914+0000","last_active":"2023-10-13T23:59:43.806655+0000","last_peered":"2023-10-13T23:59:43.806655+0000","last_clean":"2023-10-13T23:59:43.806655+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-13T23:59:43.806655+0000","last_undegraded":"2023-10-13T23:59:43.806655+0000","last_fullsized":"2023-10-13T23:59:43.806655+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:09:11.202010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-13T23:59:43.809473+0000","last_change":"2023-10-13T23:59:43.805230+0000","last_active":"2023-10-13T23:59:43.809473+0000","last_peered":"2023-10-13T23:59:43.809473+0000","last_clean":"2023-10-13T23:59:43.809473+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-13T23:59:43.809473+0000","last_undegraded":"2023-10-13T23:59:43.809473+0000","last_fullsized":"2023-10-13T23:59:43.809473+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:39:02.881760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800635+0000","last_change":"2023-10-13T23:59:43.808352+0000","last_active":"2023-10-13T23:59:44.800635+0000","last_peered":"2023-10-13T23:59:44.800635+0000","last_clean":"2023-10-13T23:59:44.800635+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-13T23:59:44.800635+0000","last_undegraded":"2023-10-13T23:59:44.800635+0000","last_fullsized":"2023-10-13T23:59:44.800635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:07:01.355189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":140,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-13T23:59:43.802099+0000","last_change":"2023-10-13T23:59:38.935251+0000","last_active":"2023-10-13T23:59:43.802099+0000","last_peered":"2023-10-13T23:59:43.802099+0000","last_clean":"2023-10-13T23:59:43.802099+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-13T23:59:43.802099+0000","last_undegraded":"2023-10-13T23:59:43.802099+0000","last_fullsized":"2023-10-13T23:59:43.802099+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_clean_scrub_stamp":"2023-10-13T23:59:37.743218+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:21:33.168386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12514,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":122880,"data_stored":33720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":624088,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":624088,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-13T23:59:46.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-13T23:59:47.222 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:47.223 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-13T23:59:47.239 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-10-13T23:59:47.218286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81544,"kb_used_data":480,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283033976,"statfs":{"total":289910292480,"available":289826791424,"internally_reserved":0,"allocated":491520,"data_stored":1285374,"data_compressed":12514,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":4782,"internal_metadata":82963794},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.434613"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800713+0000","last_change":"2023-10-13T23:59:43.806959+0000","last_active":"2023-10-13T23:59:44.800713+0000","last_peered":"2023-10-13T23:59:44.800713+0000","last_clean":"2023-10-13T23:59:44.800713+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-13T23:59:44.800713+0000","last_undegraded":"2023-10-13T23:59:44.800713+0000","last_fullsized":"2023-10-13T23:59:44.800713+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:45:57.965841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800643+0000","last_change":"2023-10-13T23:59:43.807975+0000","last_active":"2023-10-13T23:59:44.800643+0000","last_peered":"2023-10-13T23:59:44.800643+0000","last_clean":"2023-10-13T23:59:44.800643+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-13T23:59:44.800643+0000","last_undegraded":"2023-10-13T23:59:44.800643+0000","last_fullsized":"2023-10-13T23:59:44.800643+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:54:08.157126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800689+0000","last_change":"2023-10-13T23:59:43.808584+0000","last_active":"2023-10-13T23:59:44.800689+0000","last_peered":"2023-10-13T23:59:44.800689+0000","last_clean":"2023-10-13T23:59:44.800689+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-13T23:59:44.800689+0000","last_undegraded":"2023-10-13T23:59:44.800689+0000","last_fullsized":"2023-10-13T23:59:44.800689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T05:22:55.056177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800562+0000","last_change":"2023-10-13T23:59:43.807900+0000","last_active":"2023-10-13T23:59:44.800562+0000","last_peered":"2023-10-13T23:59:44.800562+0000","last_clean":"2023-10-13T23:59:44.800562+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-13T23:59:44.800562+0000","last_undegraded":"2023-10-13T23:59:44.800562+0000","last_fullsized":"2023-10-13T23:59:44.800562+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:14:06.611855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:46.526928+0000","last_change":"2023-10-13T23:59:43.806111+0000","last_active":"2023-10-13T23:59:46.526928+0000","last_peered":"2023-10-13T23:59:46.526928+0000","last_clean":"2023-10-13T23:59:46.526928+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-13T23:59:46.526928+0000","last_undegraded":"2023-10-13T23:59:46.526928+0000","last_fullsized":"2023-10-13T23:59:46.526928+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:37:07.057278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-13T23:59:43.806655+0000","last_change":"2023-10-13T23:59:43.803914+0000","last_active":"2023-10-13T23:59:43.806655+0000","last_peered":"2023-10-13T23:59:43.806655+0000","last_clean":"2023-10-13T23:59:43.806655+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-13T23:59:43.806655+0000","last_undegraded":"2023-10-13T23:59:43.806655+0000","last_fullsized":"2023-10-13T23:59:43.806655+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:09:11.202010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-13T23:59:43.809473+0000","last_change":"2023-10-13T23:59:43.805230+0000","last_active":"2023-10-13T23:59:43.809473+0000","last_peered":"2023-10-13T23:59:43.809473+0000","last_clean":"2023-10-13T23:59:43.809473+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-13T23:59:43.809473+0000","last_undegraded":"2023-10-13T23:59:43.809473+0000","last_fullsized":"2023-10-13T23:59:43.809473+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:39:02.881760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-10-13T23:59:44.800635+0000","last_change":"2023-10-13T23:59:43.808352+0000","last_active":"2023-10-13T23:59:44.800635+0000","last_peered":"2023-10-13T23:59:44.800635+0000","last_clean":"2023-10-13T23:59:44.800635+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-13T23:59:44.800635+0000","last_undegraded":"2023-10-13T23:59:44.800635+0000","last_fullsized":"2023-10-13T23:59:44.800635+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:07:01.355189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"11'97","reported_seq":140,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-10-13T23:59:43.802099+0000","last_change":"2023-10-13T23:59:38.935251+0000","last_active":"2023-10-13T23:59:43.802099+0000","last_peered":"2023-10-13T23:59:43.802099+0000","last_clean":"2023-10-13T23:59:43.802099+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-13T23:59:43.802099+0000","last_undegraded":"2023-10-13T23:59:43.802099+0000","last_fullsized":"2023-10-13T23:59:43.802099+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_clean_scrub_stamp":"2023-10-13T23:59:37.743218+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:21:33.168386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":91,"num_write":137,"num_write_kb":1894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12514,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27140,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344700,"statfs":{"total":96636764160,"available":96608972800,"internally_reserved":0,"allocated":122880,"data_stored":33720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":188416,"data_stored":627566,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27196,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344644,"statfs":{"total":96636764160,"available":96608915456,"internally_reserved":0,"allocated":180224,"data_stored":624088,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6257,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-13T23:59:47.240 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-13T23:59:47.240 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-13T23:59:47.241 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-13T23:59:47.241 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-13T23:59:47.532 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-13T23:59:47.532 INFO:teuthology.orchestra.run.smithi183.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-13T23:59:47.549 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-13T23:59:47.549 INFO:teuthology.run_tasks:Running task install... 2023-10-13T23:59:47.558 DEBUG:teuthology.task.install:project ceph 2023-10-13T23:59:47.558 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '01fff6a72a328459c1d153e5dc1de6a34e48a82f'}} 2023-10-13T23:59:47.558 DEBUG:teuthology.task.install:config {'extra_system_packages': ['fio'], 'flavor': 'default', 'sha1': '01fff6a72a328459c1d153e5dc1de6a34e48a82f'} 2023-10-13T23:59:47.558 INFO:teuthology.task.install:Using flavor: default 2023-10-13T23:59:47.564 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-13T23:59:47.564 INFO:teuthology.task.install:extra packages: [] 2023-10-13T23:59:47.565 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-key list | grep Ceph 2023-10-13T23:59:47.610 INFO:teuthology.orchestra.run.smithi183.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-13T23:59:47.633 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-13T23:59:47.633 INFO:teuthology.orchestra.run.smithi183.stdout:uid [ unknown] Ceph.com (release key) 2023-10-13T23:59:47.633 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-10-13T23:59:47.634 INFO:teuthology.task.install.deb:Installing system (non-project) packages: fio on remote deb x86_64 2023-10-13T23:59:47.634 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=01fff6a72a328459c1d153e5dc1de6a34e48a82f 2023-10-13T23:59:47.770 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default/ 2023-10-13T23:59:47.770 INFO:teuthology.task.install.deb:Package version is 18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:59:47.956 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-13T23:59:47.957 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-13T23:59:47.971 DEBUG:teuthology.orchestra.run.smithi183:> sudo apt-get update 2023-10-13T23:59:48.144 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-13T23:59:48.259 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-13T23:59:48.262 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy InRelease 2023-10-13T23:59:48.304 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:4 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy Release 2023-10-13T23:59:48.341 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-13T23:59:48.351 INFO:teuthology.orchestra.run.smithi183.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-dis-testing/01fff6a72a328459c1d153e5dc1de6a34e48a82f/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-13T23:59:48.422 INFO:teuthology.orchestra.run.smithi183.stdout:Hit:7 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-13T23:59:49.391 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:59:49.406 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6724-g01fff6a7-1jammy cephadm=18.0.0-6724-g01fff6a7-1jammy ceph-mds=18.0.0-6724-g01fff6a7-1jammy ceph-mgr=18.0.0-6724-g01fff6a7-1jammy ceph-common=18.0.0-6724-g01fff6a7-1jammy ceph-fuse=18.0.0-6724-g01fff6a7-1jammy ceph-test=18.0.0-6724-g01fff6a7-1jammy ceph-volume=18.0.0-6724-g01fff6a7-1jammy radosgw=18.0.0-6724-g01fff6a7-1jammy python3-rados=18.0.0-6724-g01fff6a7-1jammy python3-rgw=18.0.0-6724-g01fff6a7-1jammy python3-cephfs=18.0.0-6724-g01fff6a7-1jammy python3-rbd=18.0.0-6724-g01fff6a7-1jammy libcephfs2=18.0.0-6724-g01fff6a7-1jammy libcephfs-dev=18.0.0-6724-g01fff6a7-1jammy librados2=18.0.0-6724-g01fff6a7-1jammy librbd1=18.0.0-6724-g01fff6a7-1jammy rbd-fuse=18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:59:49.467 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:59:49.655 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2023-10-13T23:59:49.656 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2023-10-13T23:59:49.828 INFO:teuthology.orchestra.run.smithi183.stdout:ceph is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.828 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-common is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.829 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-fuse is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.829 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-mds is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.829 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-mgr is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.829 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-test is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.829 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-volume is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.829 INFO:teuthology.orchestra.run.smithi183.stdout:cephadm is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.830 INFO:teuthology.orchestra.run.smithi183.stdout:libcephfs-dev is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.830 INFO:teuthology.orchestra.run.smithi183.stdout:libcephfs2 is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.830 INFO:teuthology.orchestra.run.smithi183.stdout:librados2 is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.830 INFO:teuthology.orchestra.run.smithi183.stdout:librbd1 is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.830 INFO:teuthology.orchestra.run.smithi183.stdout:python3-cephfs is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.831 INFO:teuthology.orchestra.run.smithi183.stdout:python3-rados is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.831 INFO:teuthology.orchestra.run.smithi183.stdout:python3-rbd is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.831 INFO:teuthology.orchestra.run.smithi183.stdout:python3-rgw is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.831 INFO:teuthology.orchestra.run.smithi183.stdout:radosgw is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.831 INFO:teuthology.orchestra.run.smithi183.stdout:rbd-fuse is already the newest version (18.0.0-6724-g01fff6a7-1jammy). 2023-10-13T23:59:49.832 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2023-10-13T23:59:49.832 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-13T23:59:49.832 INFO:teuthology.orchestra.run.smithi183.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-13T23:59:49.832 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-13T23:59:49.832 INFO:teuthology.orchestra.run.smithi183.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-13T23:59:49.833 INFO:teuthology.orchestra.run.smithi183.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-13T23:59:49.833 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-13T23:59:49.868 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 0 newly installed, 0 to remove and 256 not upgraded. 2023-10-13T23:59:49.868 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-13T23:59:49.870 DEBUG:teuthology.orchestra.run.smithi183:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install fio 2023-10-13T23:59:49.923 INFO:teuthology.orchestra.run.smithi183.stdout:Reading package lists... 2023-10-13T23:59:50.092 INFO:teuthology.orchestra.run.smithi183.stdout:Building dependency tree... 2023-10-13T23:59:50.093 INFO:teuthology.orchestra.run.smithi183.stdout:Reading state information... 2023-10-13T23:59:50.248 INFO:teuthology.orchestra.run.smithi183.stdout:The following packages were automatically installed and are no longer required: 2023-10-13T23:59:50.248 INFO:teuthology.orchestra.run.smithi183.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libiscsi7 libpmemobj1 2023-10-13T23:59:50.248 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2-dev libpython2-stdlib libpython2.7 libpython2.7-dev 2023-10-13T23:59:50.249 INFO:teuthology.orchestra.run.smithi183.stdout: libpython2.7-minimal libpython2.7-stdlib linux-image-5.4.0-135-generic 2023-10-13T23:59:50.249 INFO:teuthology.orchestra.run.smithi183.stdout: linux-modules-5.4.0-135-generic python2 python2-dev python2-minimal 2023-10-13T23:59:50.249 INFO:teuthology.orchestra.run.smithi183.stdout: python2.7 python2.7-dev python2.7-minimal 2023-10-13T23:59:50.249 INFO:teuthology.orchestra.run.smithi183.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-13T23:59:50.250 INFO:teuthology.orchestra.run.smithi183.stdout:The following additional packages will be installed: 2023-10-13T23:59:50.250 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemblk1 2023-10-13T23:59:50.251 INFO:teuthology.orchestra.run.smithi183.stdout:Suggested packages: 2023-10-13T23:59:50.251 INFO:teuthology.orchestra.run.smithi183.stdout: gnuplot gfio python-scipy 2023-10-13T23:59:50.274 INFO:teuthology.orchestra.run.smithi183.stdout:The following NEW packages will be installed: 2023-10-13T23:59:50.275 INFO:teuthology.orchestra.run.smithi183.stdout: fio libpmemblk1 2023-10-13T23:59:54.613 INFO:teuthology.orchestra.run.smithi183.stdout:0 upgraded, 2 newly installed, 0 to remove and 256 not upgraded. 2023-10-13T23:59:54.613 INFO:teuthology.orchestra.run.smithi183.stdout:Need to get 4,112 kB of archives. 2023-10-13T23:59:54.613 INFO:teuthology.orchestra.run.smithi183.stdout:After this operation, 7,813 kB of additional disk space will be used. 2023-10-13T23:59:54.613 INFO:teuthology.orchestra.run.smithi183.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libpmemblk1 amd64 1.11.1-3build1 [65.6 kB] 2023-10-13T23:59:54.682 INFO:teuthology.orchestra.run.smithi183.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 fio amd64 3.28-1 [4,047 kB] 2023-10-13T23:59:55.189 INFO:teuthology.orchestra.run.smithi183.stdout:Fetched 4,112 kB in 5s (906 kB/s) 2023-10-13T23:59:55.349 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package libpmemblk1:amd64. 2023-10-13T23:59:55.452 INFO:teuthology.orchestra.run.smithi183.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172814 files and directories currently installed.) 2023-10-13T23:59:55.458 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../libpmemblk1_1.11.1-3build1_amd64.deb ... 2023-10-13T23:59:55.531 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking libpmemblk1:amd64 (1.11.1-3build1) ... 2023-10-13T23:59:55.867 INFO:teuthology.orchestra.run.smithi183.stdout:Selecting previously unselected package fio. 2023-10-13T23:59:55.897 INFO:teuthology.orchestra.run.smithi183.stdout:Preparing to unpack .../archives/fio_3.28-1_amd64.deb ... 2023-10-13T23:59:55.934 INFO:teuthology.orchestra.run.smithi183.stdout:Unpacking fio (3.28-1) ... 2023-10-13T23:59:56.444 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up libpmemblk1:amd64 (1.11.1-3build1) ... 2023-10-13T23:59:56.564 INFO:teuthology.orchestra.run.smithi183.stdout:Setting up fio (3.28-1) ... 2023-10-13T23:59:57.076 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-13T23:59:58.212 INFO:teuthology.orchestra.run.smithi183.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-13T23:59:58.651 INFO:teuthology.orchestra.run.smithi183.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-13T23:59:58.654 DEBUG:teuthology.parallel:result is None 2023-10-13T23:59:58.655 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=01fff6a72a328459c1d153e5dc1de6a34e48a82f 2023-10-13T23:59:58.796 DEBUG:teuthology.orchestra.run.smithi183:> dpkg-query -W -f '${Version}' ceph 2023-10-13T23:59:58.816 INFO:teuthology.orchestra.run.smithi183.stdout:18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:59:58.816 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6724-g01fff6a7-1jammy 2023-10-13T23:59:58.817 INFO:teuthology.task.install:The correct ceph version 18.0.0-6724-g01fff6a7-1jammy is installed. 2023-10-13T23:59:58.817 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2023-10-13T23:59:58.817 INFO:teuthology.run_tasks:Running task workunit... 2023-10-13T23:59:58.831 INFO:tasks.workunit:Pulling workunits from ref 735f68d7bf91d643813a8420fc2b11441f4bd8c4 2023-10-13T23:59:58.832 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-10-13T23:59:58.832 DEBUG:teuthology.orchestra.run.smithi183:> stat -- /home/ubuntu/cephtest/mnt.0 2023-10-13T23:59:58.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-13T23:59:58.863 INFO:teuthology.orchestra.run.smithi183.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-10-13T23:59:58.863 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-10-13T23:59:58.910 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-10-13T23:59:58.910 DEBUG:teuthology.orchestra.run.smithi183:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-10-13T23:59:58.958 INFO:tasks.workunit:timeout=3h 2023-10-13T23:59:58.959 INFO:tasks.workunit:cleanup=True 2023-10-13T23:59:58.959 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/idryomov/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 735f68d7bf91d643813a8420fc2b11441f4bd8c4 2023-10-13T23:59:59.008 INFO:tasks.workunit.client.0.smithi183.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-10-14T00:00:50.007 INFO:tasks.workunit.client.0.smithi183.stderr:Note: switching to '735f68d7bf91d643813a8420fc2b11441f4bd8c4'. 2023-10-14T00:00:50.008 INFO:tasks.workunit.client.0.smithi183.stderr: 2023-10-14T00:00:50.008 INFO:tasks.workunit.client.0.smithi183.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-10-14T00:00:50.008 INFO:tasks.workunit.client.0.smithi183.stderr:changes and commit them, and you can discard any commits you make in this 2023-10-14T00:00:50.008 INFO:tasks.workunit.client.0.smithi183.stderr:state without impacting any branches by switching back to a branch. 2023-10-14T00:00:50.009 INFO:tasks.workunit.client.0.smithi183.stderr: 2023-10-14T00:00:50.009 INFO:tasks.workunit.client.0.smithi183.stderr:If you want to create a new branch to retain commits you create, you may 2023-10-14T00:00:50.009 INFO:tasks.workunit.client.0.smithi183.stderr:do so (now or later) by using -c with the switch command. Example: 2023-10-14T00:00:50.009 INFO:tasks.workunit.client.0.smithi183.stderr: 2023-10-14T00:00:50.009 INFO:tasks.workunit.client.0.smithi183.stderr: git switch -c 2023-10-14T00:00:50.010 INFO:tasks.workunit.client.0.smithi183.stderr: 2023-10-14T00:00:50.010 INFO:tasks.workunit.client.0.smithi183.stderr:Or undo this operation with: 2023-10-14T00:00:50.010 INFO:tasks.workunit.client.0.smithi183.stderr: 2023-10-14T00:00:50.010 INFO:tasks.workunit.client.0.smithi183.stderr: git switch - 2023-10-14T00:00:50.010 INFO:tasks.workunit.client.0.smithi183.stderr: 2023-10-14T00:00:50.011 INFO:tasks.workunit.client.0.smithi183.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-10-14T00:00:50.011 INFO:tasks.workunit.client.0.smithi183.stderr: 2023-10-14T00:00:50.011 INFO:tasks.workunit.client.0.smithi183.stderr:HEAD is now at 735f68d7bf9 Merge branch 'wip-62891' of https://github.com/ajarr/ceph into HEAD 2023-10-14T00:00:50.013 DEBUG:teuthology.orchestra.run.smithi183:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-10-14T00:00:50.066 INFO:tasks.workunit.client.0.smithi183.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-10-14T00:00:50.069 INFO:tasks.workunit.client.0.smithi183.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-14T00:00:50.069 INFO:tasks.workunit.client.0.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-10-14T00:00:50.879 INFO:tasks.workunit.client.0.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-10-14T00:00:50.967 INFO:tasks.workunit.client.0.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-10-14T00:00:51.017 INFO:tasks.workunit.client.0.smithi183.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-10-14T00:00:51.018 INFO:tasks.workunit.client.0.smithi183.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-14T00:00:51.019 INFO:tasks.workunit.client.0.smithi183.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-10-14T00:00:51.065 INFO:tasks.workunit.client.0.smithi183.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-10-14T00:00:51.069 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2023-10-14T00:00:51.069 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-10-14T00:00:51.117 INFO:tasks.workunit:Running workunits matching rbd/rbd_support_module_recovery.sh on client.0... 2023-10-14T00:00:51.118 INFO:tasks.workunit:Running workunit rbd/rbd_support_module_recovery.sh... 2023-10-14T00:00:51.119 DEBUG:teuthology.orchestra.run.smithi183:workunit test rbd/rbd_support_module_recovery.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=735f68d7bf91d643813a8420fc2b11441f4bd8c4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd_support_module_recovery.sh 2023-10-14T00:00:51.171 INFO:tasks.workunit.client.0.smithi183.stderr:+ POOL=rbd 2023-10-14T00:00:51.171 INFO:tasks.workunit.client.0.smithi183.stderr:+ IMAGE_PREFIX=image 2023-10-14T00:00:51.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ NUM_IMAGES=20 2023-10-14T00:00:51.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ RUN_TIME=3600 2023-10-14T00:00:51.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror pool enable rbd image 2023-10-14T00:00:51.222 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror pool peer add rbd dummy 2023-10-14T00:00:51.259 INFO:tasks.workunit.client.0.smithi183.stdout:962dc040-ef6d-4492-8e69-1969de744039 2023-10-14T00:00:51.262 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i = 1 )) 2023-10-14T00:00:51.262 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:51.263 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image1 2023-10-14T00:00:52.678 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image1 snapshot 2023-10-14T00:00:53.695 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:00:53.710 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 1m 2023-10-14T00:00:53.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:00:53.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:53.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image2 2023-10-14T00:00:53.824 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image2 snapshot 2023-10-14T00:00:54.691 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:00:54.704 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image2 1m 2023-10-14T00:00:54.766 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:00:54.767 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:54.767 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image3 2023-10-14T00:00:54.816 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image3 snapshot 2023-10-14T00:00:55.709 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:00:55.723 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image3 1m 2023-10-14T00:00:55.783 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:00:55.783 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:55.783 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image4 2023-10-14T00:00:55.833 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image4 snapshot 2023-10-14T00:00:56.720 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:00:56.734 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image4 1m 2023-10-14T00:00:56.799 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:00:56.800 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:56.800 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image5 2023-10-14T00:00:56.848 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image5 snapshot 2023-10-14T00:00:57.720 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:00:57.734 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image5 1m 2023-10-14T00:00:57.795 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:00:57.795 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:57.796 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image6 2023-10-14T00:00:57.845 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image6 snapshot 2023-10-14T00:00:58.740 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:00:58.758 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image6 1m 2023-10-14T00:00:58.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:00:58.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:58.822 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image7 2023-10-14T00:00:58.871 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image7 snapshot 2023-10-14T00:00:59.748 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:00:59.759 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image7 1m 2023-10-14T00:00:59.819 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:00:59.819 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:00:59.820 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image8 2023-10-14T00:00:59.868 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image8 snapshot 2023-10-14T00:01:00.757 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:00.768 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image8 1m 2023-10-14T00:01:00.827 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:00.828 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:00.828 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image9 2023-10-14T00:01:00.874 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image9 snapshot 2023-10-14T00:01:01.606 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:01.616 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image9 1m 2023-10-14T00:01:01.675 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:01.676 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:01.676 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image10 2023-10-14T00:01:01.723 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image10 snapshot 2023-10-14T00:01:02.768 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:02.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image10 1m 2023-10-14T00:01:02.841 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:02.841 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:02.841 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image11 2023-10-14T00:01:02.892 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image11 snapshot 2023-10-14T00:01:03.781 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:03.793 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image11 1m 2023-10-14T00:01:03.854 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:03.854 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:03.854 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image12 2023-10-14T00:01:03.901 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image12 snapshot 2023-10-14T00:01:04.783 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:04.797 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image12 1m 2023-10-14T00:01:04.859 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:04.859 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:04.860 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image13 2023-10-14T00:01:04.910 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image13 snapshot 2023-10-14T00:01:05.792 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:05.805 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image13 1m 2023-10-14T00:01:05.867 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:05.867 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:05.867 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image14 2023-10-14T00:01:05.918 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image14 snapshot 2023-10-14T00:01:06.800 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:06.813 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image14 1m 2023-10-14T00:01:06.874 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:06.875 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:06.875 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image15 2023-10-14T00:01:06.925 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image15 snapshot 2023-10-14T00:01:07.817 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:07.829 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image15 1m 2023-10-14T00:01:07.893 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:07.894 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:07.894 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image16 2023-10-14T00:01:07.940 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image16 snapshot 2023-10-14T00:01:08.820 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:08.835 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image16 1m 2023-10-14T00:01:08.899 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:08.899 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:08.899 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image17 2023-10-14T00:01:08.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image17 snapshot 2023-10-14T00:01:09.831 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:09.845 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image17 1m 2023-10-14T00:01:09.904 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:09.904 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:09.905 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image18 2023-10-14T00:01:09.952 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image18 snapshot 2023-10-14T00:01:10.838 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:10.852 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image18 1m 2023-10-14T00:01:10.914 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:10.915 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:10.915 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image19 2023-10-14T00:01:10.964 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image19 snapshot 2023-10-14T00:01:11.842 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:11.853 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image19 1m 2023-10-14T00:01:11.912 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:11.913 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:11.913 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd create -s 1G --image-feature exclusive-lock rbd/image20 2023-10-14T00:01:11.959 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror image enable rbd/image20 snapshot 2023-10-14T00:01:12.853 INFO:tasks.workunit.client.0.smithi183.stdout:Mirroring enabled 2023-10-14T00:01:12.866 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image20 1m 2023-10-14T00:01:12.927 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:12.927 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:12.927 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i = 1 )) 2023-10-14T00:01:12.928 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:12.928 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image1 2023-10-14T00:01:13.052 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd0 2023-10-14T00:01:13.052 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.052 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.052 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd0 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.053 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image2 2023-10-14T00:01:13.124 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd1 2023-10-14T00:01:13.124 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.124 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.124 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd1 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.125 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image3 2023-10-14T00:01:13.197 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd2 2023-10-14T00:01:13.198 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.198 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.198 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd2 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.199 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image4 2023-10-14T00:01:13.273 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd3 2023-10-14T00:01:13.273 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.274 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.274 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd3 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.274 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image5 2023-10-14T00:01:13.341 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd4 2023-10-14T00:01:13.342 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.342 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.342 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd4 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.342 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image6 2023-10-14T00:01:13.435 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd5 2023-10-14T00:01:13.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd5 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.436 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image7 2023-10-14T00:01:13.540 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd6 2023-10-14T00:01:13.540 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.540 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.541 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd6 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.541 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image8 2023-10-14T00:01:13.663 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd7 2023-10-14T00:01:13.664 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.664 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.665 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd7 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.665 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image9 2023-10-14T00:01:13.813 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd8 2023-10-14T00:01:13.813 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.814 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.814 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd8 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.819 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image10 2023-10-14T00:01:13.952 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd9 2023-10-14T00:01:13.952 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:13.953 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:13.953 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd9 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:13.954 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image11 2023-10-14T00:01:14.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:14.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:14.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:14.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:14.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:14.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:14.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:14.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:14.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:14.085+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:14.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd10 2023-10-14T00:01:14.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:14.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:14.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd10 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:14.119 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image12 2023-10-14T00:01:14.296 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd11 2023-10-14T00:01:14.297 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:14.297 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:14.297 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd11 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:14.297 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image13 2023-10-14T00:01:14.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd12 2023-10-14T00:01:14.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:14.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:14.440 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image14 2023-10-14T00:01:14.442 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd12 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:14.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd13 2023-10-14T00:01:14.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:14.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:14.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd13 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:14.625 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image15 2023-10-14T00:01:14.797 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd14 2023-10-14T00:01:14.797 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:14.798 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:14.798 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image16 2023-10-14T00:01:14.799 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd14 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:15.009 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd15 2023-10-14T00:01:15.010 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:15.010 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:15.011 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd15 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:15.011 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image17 2023-10-14T00:01:15.136 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd16 2023-10-14T00:01:15.137 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:15.137 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:15.137 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd16 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:15.138 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image18 2023-10-14T00:01:15.281 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd17 2023-10-14T00:01:15.281 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:15.282 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:15.282 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd17 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:15.287 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image19 2023-10-14T00:01:15.428 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd18 2023-10-14T00:01:15.428 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:15.429 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:15.430 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd18 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:15.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:15.433+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:15.454 INFO:tasks.workunit.client.0.smithi183.stderr:++ sudo rbd device map rbd/image20 2023-10-14T00:01:15.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ DEVS[$i]=/dev/rbd19 2023-10-14T00:01:15.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T00:01:15.657 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 20 )) 2023-10-14T00:01:15.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ fio --name=fiotest --filename=/dev/rbd19 --rw=randrw --bs=4K --direct=1 --ioengine=libaio --iodepth=2 --runtime=43200 --time_based 2023-10-14T00:01:15.659 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:01:15.665 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241675 2023-10-14T00:01:15.665 INFO:tasks.workunit.client.0.smithi183.stderr:+ END_TIME=1697245275 2023-10-14T00:01:15.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR= 2023-10-14T00:01:15.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR= 2023-10-14T00:01:15.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:01:15.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n '' ]] 2023-10-14T00:01:15.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:01:15.988 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:15.981+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:16.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:16.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:16.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:16.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:16.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:16.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:16.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:16.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:16.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:16.309+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:16.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:16.309+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:16.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:16.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:18.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.221+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:18.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:18.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:18.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:18.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:18.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:18.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.233+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:18.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.233+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:18.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.233+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:18.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:18.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:18.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:18.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.977+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:18.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:18.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:18.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:18.977+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:19.035 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:19.029+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:19.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:19.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:19.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:19.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:19.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:19.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:19.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.217+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:19.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:19.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.225+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:19.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:19.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:19.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:19.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:19.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.237+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:19.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.237+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:19.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.237+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:19.435 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:19.433+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:19.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:19.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:20.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:20.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:19.997+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:20.173 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:20.169+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:21.213 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:21.209+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:21.214 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:21.213+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:21.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.437+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:21.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:21.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:21.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:21.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:21.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:21.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.445+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:21.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.445+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:21.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:21.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:21.719 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:21.717+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:22.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:22.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:22.181 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:22.177+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:22.183 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:22.177+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:23.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:23.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:23.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:23.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:23.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:23.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:23.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:23.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:23.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:23.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:23.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:23.461+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:24.087 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:24.081+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:01:24.087 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:24.085+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:24.087 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:24.085+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:24.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:24.297+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:01:24.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:24.297+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:24.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:24.297+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:24.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:24.297+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:25.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:25.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:25.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:25.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.276+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:25.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.276+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:25.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.320+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:01:25.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:25.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:25.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:25.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:25.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:25.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:25.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:25.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:25.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:25.324+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:25.674 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:01:25.674 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:01:25.679 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:01:25.691 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:01:25.699 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:25.692+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:26.761 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:26.756+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:27.078 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2449655932 2023-10-14T00:01:27.078 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:01:27.083 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241687 2023-10-14T00:01:27.083 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:01:27.083 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2449655932 ]] 2023-10-14T00:01:27.083 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2449655932 != '' ]] 2023-10-14T00:01:27.084 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2449655932 2023-10-14T00:01:27.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:27.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.132+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:27.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:27.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:27.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:27.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.164+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:27.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.164+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:27.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.176+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:27.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:27.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:27.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:27.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:27.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:27.488+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:27.625 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:27.620+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:27.633 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:27.628+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:28.681 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:28.676+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:28.695 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:28.692+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:29.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.116+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:01:29.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.116+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:29.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:29.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:29.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:29.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:29.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:29.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.120+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:29.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.120+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:29.399 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:29.396+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:01:29.421 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:29.416+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:29.422 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:29.416+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:29.422 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:29.416+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:29.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:29.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:29.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:29.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:29.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:29.552+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:29.602 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2449655932 until 2023-10-14T01:01:28.836806+0000 (3600 sec) 2023-10-14T00:01:29.677 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:01:29.680 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2449655932 2023-10-14T00:01:30.306 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:30.300+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:30.649 INFO:tasks.workunit.client.0.smithi183.stderr:listed 1 entries 2023-10-14T00:01:30.685 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2449655932 2023-10-14T00:01:30.686 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:01:30.717 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:30.712+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:31.245 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:31.240+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:31.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:31.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:31.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:31.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:31.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:31.388+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:31.652 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:31.644+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:31.657 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:31.652+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:31.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:31.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:31.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:31.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:31.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:31.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:31.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:31.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:32.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:32.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:32.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:32.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.248+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:32.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.248+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:32.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.248+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:32.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.388+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:32.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:32.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:32.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:32.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:32.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:32.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.420+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:32.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.420+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:32.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.424+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:32.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.704+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:32.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:32.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:32.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:32.716+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:33.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:33.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:33.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:33.764+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:34.778 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:01:34.772+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:01:34.778 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:01:34.772+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:01:34.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:34.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:34.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:34.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:34.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:34.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:34.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:34.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:34.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:34.872+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:34.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:34.872+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:34.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:34.880+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:35.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.148+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:35.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:35.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:35.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:35.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:35.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:35.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:35.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:35.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:35.222 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:35.216+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:35.224 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:35.220+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:35.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.356+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:35.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:35.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:35.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:35.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:35.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:35.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:35.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:35.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.388+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:35.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:35.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:35.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:35.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:35.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.444+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:35.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.456+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:35.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.460+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:35.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:35.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:35.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:35.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:35.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.640+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:35.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.640+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:35.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:35.640+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:36.109 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:36.104+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:36.200 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:36.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:36.201 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:36.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:36.201 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:36.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:36.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.732+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:01:36.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.732+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:01:36.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.732+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:36.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:36.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:36.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:36.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:36.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:36.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.744+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:36.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.744+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:36.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:36.748+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:36.752 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:36.748+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:36.778 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:36.772+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:36.779 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:36.776+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:37.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.248+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:37.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:37.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:37.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:37.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:37.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:37.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.284+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:37.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.284+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:37.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:37.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:37.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:37.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:37.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:38.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:38.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:38.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:38.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:38.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.028+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:38.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.208+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:38.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:38.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:38.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:38.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:38.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:38.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.232+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:38.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.232+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:38.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:38.240+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:39.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:39.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:39.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:39.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:39.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.212+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:39.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.588+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:39.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:39.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:39.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:39.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:39.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.600+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:39.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.600+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:39.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:39.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.876+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:39.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:39.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:39.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:39.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:39.888+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:40.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.124+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:40.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.132+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:40.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:40.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:40.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:40.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.144+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:40.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.144+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:40.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.144+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:40.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:40.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:40.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:40.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:40.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:40.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.536+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:40.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.536+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:40.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:40.536+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:40.690 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:01:40.690 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:01:40.692 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:01:40.723 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:01:41.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:41.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:41.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:41.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:41.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:41.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:41.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:41.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:41.424+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:42.162 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2122915279 2023-10-14T00:01:42.162 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:01:42.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241702 2023-10-14T00:01:42.166 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:01:42.166 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2122915279 ]] 2023-10-14T00:01:42.166 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2122915279 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\4\9\6\5\5\9\3\2 ]] 2023-10-14T00:01:42.166 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2122915279 2023-10-14T00:01:42.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:42.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:42.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:42.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.408+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:42.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.468+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:01:42.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.468+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:01:42.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.468+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:42.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:42.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:42.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:42.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:42.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:42.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:42.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:42.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:42.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.652+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:42.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.652+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:42.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:42.652+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:42.950 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:42.948+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:44.659 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2122915279 until 2023-10-14T01:01:44.020926+0000 (3600 sec) 2023-10-14T00:01:44.722 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2122915279 2023-10-14T00:01:44.726 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:01:44.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:44.894 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:44.892+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:44.895 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:44.892+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:44.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:44.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:44.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:44.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:44.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:44.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:44.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:44.912+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:45.135 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:45.132+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:45.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:45.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:45.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:45.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:45.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:45.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.264+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:45.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.264+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:45.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.264+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:45.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:45.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:45.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:45.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:45.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:45.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:45.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:45.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:45.384+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:45.859 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:45.852+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:45.866 INFO:tasks.workunit.client.0.smithi183.stderr:listed 2 entries 2023-10-14T00:01:45.945 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2122915279 2023-10-14T00:01:45.945 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:01:46.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:46.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:46.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:46.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:46.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:46.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:46.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:46.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.192+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:46.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:46.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:46.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:46.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:46.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.344+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:46.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.344+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:46.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.352+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:46.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.564+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:46.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.636+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:46.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:46.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:46.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:46.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:46.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:46.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.672+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:46.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.672+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:46.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:46.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:47.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:46.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:47.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:47.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:47.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.020+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:47.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:47.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:47.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:47.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:47.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.444+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:47.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.444+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:47.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:47.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:48.488 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:48.488+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:48.983 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:48.984+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:49.004 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:49.004+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:49.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:49.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:49.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:49.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:49.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.635+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:49.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.635+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:49.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.639+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:49.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:49.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:49.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:50.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:50.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:50.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:50.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:50.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:50.015+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:50.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:50.015+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:50.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:50.019+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:51.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:51.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:51.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:51.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:51.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.415+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:51.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:51.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:51.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:51.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:51.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.675+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:51.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.675+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:51.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:51.675+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:52.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.307+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:52.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:52.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:52.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:52.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:52.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:52.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.383+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:52.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.383+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:52.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.411+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:52.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:52.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.875+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:01:52.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.875+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:01:52.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:52.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:52.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:52.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:52.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:52.919+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:53.842 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:53.839+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:53.854 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:53.851+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:54.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:54.987+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:54.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:54.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:55.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:55.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:55.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:55.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:55.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.099+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:55.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.099+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:55.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.103+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:55.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.119+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:01:55.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.127+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:01:55.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.127+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:55.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:55.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:55.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:55.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:55.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:55.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.143+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:55.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.143+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:55.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:55.143+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:55.952 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:01:55.953 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:01:55.954 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:01:55.955 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:01:56.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:56.034 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:56.031+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:01:56.061 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:56.059+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:01:56.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:56.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:56.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:56.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.111+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:56.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.111+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:56.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.115+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:56.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:56.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:56.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:56.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:56.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.423+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:56.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:56.423+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:56.669 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:56.667+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:01:57.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.199+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:01:57.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:01:57.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:01:57.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:01:57.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:01:57.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:01:57.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.247+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:01:57.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.247+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:01:57.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:01:57.247+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:01:57.482 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2122915279 2023-10-14T00:01:57.484 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:01:57.485 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241717 2023-10-14T00:01:57.486 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:01:57.486 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2122915279 ]] 2023-10-14T00:01:57.486 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2122915279 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\1\2\2\9\1\5\2\7\9 ]] 2023-10-14T00:01:57.486 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:01:57.817 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:01:57.811+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.023+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.023+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb1e6dc80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.027+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.027+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275ac00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.028 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.027+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.029 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.027+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb295c300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.035+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.035+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb275b500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.039+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.039+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb295d680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.044 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.043+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.044 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.043+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275a900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.056 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.055+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.056 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.055+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb1e6c900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.067+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.067+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2930180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.079+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.079+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb1e6dc00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.079+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.079+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275af80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.079+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.079+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275b880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.091+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.091+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2931900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.095+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.095+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb295d700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.104 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.103+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.104 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.103+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb295d780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.107+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.107+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb295d880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.119+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.119+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275b300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275a900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275af00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb295de00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.134 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:00.134 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:02:00.131+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb295dc80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:02:01.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:01.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:01.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:01.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:01.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:01.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:01.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:01.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:01.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:01.355+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:01.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:01.355+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:01.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:01.359+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:01.914 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:01.911+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:02:01.920 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:01.919+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:01.921 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:01.919+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:02.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:02.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:02.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:02.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:02.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:02.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:02.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:02.339+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:03.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.391+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:03.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:03.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:03.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:03.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.407+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:03.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:03.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:03.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:03.527+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:04.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:04.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:04.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:04.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.231+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:04.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.747+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:04.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:04.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:04.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:04.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:04.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:04.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:04.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:04.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.831+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:04.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:04.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:04.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:04.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:04.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:04.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:04.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:05.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.263+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:05.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.263+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:05.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.263+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:05.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:05.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:05.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:05.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:05.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:05.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.307+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:05.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.307+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:05.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:05.311+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:07.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.223+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:07.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.223+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:07.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.223+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:07.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:07.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:07.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:07.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:07.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:07.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.271+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:07.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.271+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:07.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.271+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:07.490 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:02:07.490 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:02:07.495 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:02:07.500 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:02:07.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.695+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:07.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:07.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:07.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:07.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:07.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:07.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.743+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:07.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.743+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:07.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:07.743+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:08.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.455+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:08.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.455+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:08.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.455+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:08.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:08.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:08.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:08.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:08.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.507+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:08.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.507+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:08.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:08.507+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:09.014 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2122915279 2023-10-14T00:02:09.014 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:02:09.015 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241729 2023-10-14T00:02:09.015 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:02:09.015 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2122915279 ]] 2023-10-14T00:02:09.016 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2122915279 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\1\2\2\9\1\5\2\7\9 ]] 2023-10-14T00:02:09.016 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:02:09.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.623+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:09.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:09.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:09.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:09.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:09.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.647+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:09.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.647+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:09.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.647+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:09.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.907+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:09.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:09.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:09.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:09.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:09.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:09.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.939+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:09.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.939+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:09.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:09.939+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:10.691 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:10.687+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:02:10.696 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:10.695+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:10.705 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:10.703+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:10.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:10.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:10.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:10.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:10.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:10.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:10.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:10.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:10.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:10.959+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:10.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:10.959+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:10.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:10.967+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:11.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:11.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:11.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:11.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:11.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:11.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.503+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:11.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.503+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:11.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:11.503+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:12.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.359+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:12.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.359+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:12.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:12.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:12.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:12.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:12.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:12.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:12.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:12.387+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:13.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:13.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:13.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.302+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:13.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:13.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:13.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.322+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:13.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.322+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:13.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.334+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:13.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.702+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:13.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:13.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:13.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.746+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.746+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:13.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.750+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:13.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:13.977 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:13.974+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:02:13.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:13.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:13.986+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:13.992 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:13.990+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:13.998 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:13.994+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:14.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.014+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:14.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:14.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:14.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:14.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:14.407 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:14.406+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:14.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.666+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:14.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.670+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:14.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:14.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:14.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:14.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:14.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:14.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.698+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:14.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.698+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:14.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:14.698+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:15.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.218+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:15.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:15.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:15.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:15.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:15.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:15.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.238+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:15.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.238+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:15.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:15.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:15.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:15.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:15.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:15.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.482+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:15.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.482+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:15.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:15.490+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:15.543 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:15.542+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:15.548 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:15.546+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:16.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:16.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:16.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:16.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:16.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.598+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:16.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.598+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:16.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.602+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:16.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.774+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:16.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:16.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:16.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:16.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:16.830+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:17.138 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:17.134+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:02:17.184 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:17.182+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:17.185 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:17.182+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:17.254 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:02:17.250+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:02:17.255 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:02:17.250+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:02:18.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:18.834+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:18.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:18.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:18.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:18.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:18.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:18.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:18.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:18.862+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:19.017 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:02:19.022 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:02:19.027 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:02:19.033 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:02:20.347 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2122915279 2023-10-14T00:02:20.353 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:02:20.354 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241740 2023-10-14T00:02:20.354 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:02:20.354 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2122915279 ]] 2023-10-14T00:02:20.354 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2122915279 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\1\2\2\9\1\5\2\7\9 ]] 2023-10-14T00:02:20.355 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:02:20.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.706+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:20.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.706+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:20.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.714+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:20.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:20.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:20.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:20.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:20.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:20.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.738+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:20.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.738+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:20.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:20.738+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:22.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.262+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:22.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:22.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:22.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:22.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:22.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:22.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.298+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:22.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.302+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:22.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.306+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:22.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.646+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:22.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:22.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:22.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:22.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:22.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.750+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:22.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.750+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:22.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:22.750+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:23.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.022+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:23.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:23.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:23.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:23.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:23.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.038+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:23.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.402+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:23.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:23.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:23.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:23.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:23.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.462+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:23.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.462+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:23.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:23.466+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:24.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.462+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:24.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:24.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:24.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:24.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:24.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:24.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.518+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:24.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.518+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:24.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:24.518+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:26.810 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:26.806+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:27.617 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:27.614+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:28.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:28.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:28.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:28.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:28.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:28.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:28.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:28.546+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:28.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:28.546+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:29.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:29.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:29.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:29.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:29.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:29.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.058+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:29.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.058+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:29.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.058+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:29.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:29.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:29.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:29.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:29.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.250+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:29.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.250+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:29.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:29.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:30.360 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:02:30.360 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:02:30.360 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:02:30.363 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:02:30.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:30.830+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:30.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:30.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:30.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:30.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:30.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:30.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:30.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:30.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:30.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:30.870+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:31.836 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2122915279 2023-10-14T00:02:31.844 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:02:31.845 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241751 2023-10-14T00:02:31.846 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:02:31.846 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2122915279 ]] 2023-10-14T00:02:31.846 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2122915279 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\1\2\2\9\1\5\2\7\9 ]] 2023-10-14T00:02:31.846 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:02:32.642 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:32.638+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:02:32.656 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:32.654+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:32.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.762+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:32.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:32.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:32.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:32.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:32.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.810+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:32.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.822+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:32.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.830+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:32.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:32.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:32.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:32.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:32.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.898+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:32.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:32.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:32.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:32.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:33.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:33.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:33.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:33.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:33.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:33.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:33.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:33.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:33.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:33.090+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:33.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:33.090+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:33.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:33.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:33.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:33.398+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:33.671 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:33.670+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:34.134 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:34.130+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:34.580 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:34.578+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:34.594 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:34.590+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:34.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:34.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:34.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:34.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.706+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:34.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.706+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:34.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:34.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:34.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:34.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:35.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:35.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:35.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:35.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:35.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:35.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:35.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:35.858+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:35.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:35.858+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:35.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:35.882+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:36.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.462+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:36.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:36.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:36.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:36.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:36.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:36.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.482+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:36.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.482+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:36.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.486+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:36.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.550+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:36.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:36.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:36.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:36.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:36.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.574+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:36.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.574+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:36.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.578+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:36.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:36.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:36.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:36.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:36.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:36.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.950+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:36.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.950+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:36.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:36.958+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:37.176 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:37.174+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:38.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:38.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:38.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:38.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:38.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:38.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:38.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:38.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:38.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:38.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:38.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:38.706+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:38.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:38.706+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:39.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:39.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:39.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:39.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:39.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.494+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:39.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.494+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:39.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.518+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:39.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.758+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:39.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:39.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:39.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:39.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:39.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:39.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.794+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:39.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.794+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:39.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.794+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:39.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:39.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:39.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.858+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:39.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.858+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:39.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:39.858+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:40.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.918+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:40.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:40.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:40.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:40.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:40.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:40.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.950+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:40.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.950+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:40.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:40.950+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:41.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:41.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:41.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:41.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:41.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.546+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:41.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:41.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.546+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:41.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:41.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:41.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:41.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:41.863 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:02:41.867 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:02:41.869 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:02:41.881 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:02:42.344 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:42.342+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:02:42.344 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:42.342+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:43.112 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2122915279 2023-10-14T00:02:43.112 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:02:43.112 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241763 2023-10-14T00:02:43.113 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:02:43.113 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2122915279 ]] 2023-10-14T00:02:43.113 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2122915279 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\1\2\2\9\1\5\2\7\9 ]] 2023-10-14T00:02:43.113 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:02:47.572 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:47.570+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:47.579 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:47.578+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:47.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.826+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:47.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.826+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:47.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.834+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:47.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:47.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:47.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:47.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:47.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:47.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.894+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:47.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.894+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:47.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.898+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:47.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.898+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:47.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.898+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:47.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.898+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:47.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:47.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:47.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:47.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:47.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:47.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.918+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:47.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.918+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:47.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:47.918+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:48.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:48.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:48.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:48.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:48.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:48.342+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:49.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:49.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:49.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:49.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:49.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:49.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:49.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:49.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:49.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:49.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:49.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:49.502+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:49.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:49.502+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:50.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:50.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:50.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:50.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:50.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.069+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:50.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.069+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:50.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.069+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:50.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:50.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:50.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:50.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:50.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.449+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:50.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.449+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:50.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:50.457+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:50.754 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:50.749+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:51.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:51.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:51.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:51.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:51.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.381+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:51.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.393+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:51.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.393+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:51.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.569+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:51.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:51.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:51.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:51.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:51.573+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:52.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:52.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:52.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:52.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:52.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:52.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:52.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:52.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:52.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:52.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:52.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:52.201+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:52.277 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:52.273+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:02:52.308 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:52.301+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:53.121 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:02:53.123 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:02:53.139 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:02:53.150 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:02:53.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.609+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:53.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:53.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:53.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:53.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:53.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:53.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.657+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:53.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.657+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:53.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:53.657+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:54.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:54.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:54.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:54.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:54.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:54.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:54.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:54.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:54.181+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:54.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3703893601 2023-10-14T00:02:54.731 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:02:54.733 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241774 2023-10-14T00:02:54.733 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:02:54.734 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3703893601 ]] 2023-10-14T00:02:54.734 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3703893601 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\1\2\2\9\1\5\2\7\9 ]] 2023-10-14T00:02:54.734 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/3703893601 2023-10-14T00:02:54.740 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:54.737+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:54.758 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:54.753+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:55.199 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:55.193+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:55.283 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:55.277+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:55.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.597+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:02:55.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:55.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:55.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:55.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:55.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:55.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.637+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:55.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.637+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:55.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:55.653+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:56.739 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:56.737+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:02:56.740 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:56.737+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:56.740 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:56.737+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:02:57.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:57.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:57.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:57.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:57.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:57.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.153+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:57.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.153+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:57.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:57.157+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:57.624 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/3703893601 until 2023-10-14T01:02:56.956147+0000 (3600 sec) 2023-10-14T00:02:57.676 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:02:57.687 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/3703893601 2023-10-14T00:02:58.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:58.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:58.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:58.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:58.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.185+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:58.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.185+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:58.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.197+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:58.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:58.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:58.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:58.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:58.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.653+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:58.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.653+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:58.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:58.653+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:58.836 INFO:tasks.workunit.client.0.smithi183.stderr:listed 3 entries 2023-10-14T00:02:58.885 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:58.877+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:02:58.891 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/3703893601 2023-10-14T00:02:58.892 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:02:58.900 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:02:58.897+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:02:59.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:59.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:59.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:59.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:59.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:59.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:59.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:02:59.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.837+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:02:59.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.837+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:02:59.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.837+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:02:59.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:02:59.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:02:59.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:02:59.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:02:59.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:02:59.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.861+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:02:59.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:02:59.861+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2931580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295d700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295d880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.025+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295c280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.039 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.033+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.039 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.033+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c3a100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.057 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.053+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.057 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.053+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c3a300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.057 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.053+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.058 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.053+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c3a500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.061+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.061+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2931d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.061+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.061+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb1e1aa00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.073+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.073+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb1e1a480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.073+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.073+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb25d5c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.089+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.089+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.089+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.097 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.089+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.098 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.089+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.098 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.093+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295c980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.093+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295d680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.093+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2931580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.093+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c3a180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.093+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295c900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.101+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.101+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295d900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.101+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.101+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.101+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccab98c680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.101+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c3a080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.105+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:03:00.105+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c3a280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:03:00.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:00.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:00.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:00.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:00.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:00.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:00.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:00.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:00.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:00.877+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:01.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:01.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:01.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:01.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:01.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:01.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.317+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:01.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.317+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:01.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.341+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:01.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.841+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:01.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:01.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:01.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:01.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:01.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:01.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.877+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:01.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.877+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:01.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:01.877+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:01.971 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:01.961+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:01.980 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:01.977+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:02.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:02.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:02.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:02.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:02.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.225+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:02.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:02.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:02.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:02.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.877+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:02.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.901+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:02.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:02.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:02.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:02.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:02.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:02.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:02.949+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:03.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:03.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:03.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:03.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:03.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:03.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.313+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:03.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.313+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:03.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:03.325+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:03.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:03.357+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:03.370 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:03.365+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:04.417 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:04.413+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:04.419 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:04.417+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:04.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:04.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:04.889 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:04.885+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:05.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.377+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:03:05.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.389+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:05.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:05.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:05.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:05.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:05.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:05.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.445+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:05.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.445+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:05.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:05.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:05.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.561+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:05.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:05.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:05.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.565+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:05.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.565+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:05.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:05.585+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:06.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:06.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:06.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:06.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:06.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:06.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:06.895 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:06.889+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:06.901 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:06.897+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:07.897 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:07.893+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:03:07.898 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:07.893+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:07.898 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:07.893+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:07.898 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:07.893+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:08.908 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:03:08.909 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:03:08.909 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:03:08.909 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:03:09.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:09.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:09.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:09.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:09.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:09.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.205+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:09.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.205+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:09.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.205+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:09.795 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:09.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:03:09.797 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:09.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:09.801 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:09.797+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:09.811 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:09.809+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:09.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.945+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:09.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:09.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:09.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:09.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:09.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:09.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:09.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:10.261 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3703893601 2023-10-14T00:03:10.262 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:03:10.264 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241790 2023-10-14T00:03:10.264 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:03:10.264 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3703893601 ]] 2023-10-14T00:03:10.265 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3703893601 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\0\3\8\9\3\6\0\1 ]] 2023-10-14T00:03:10.265 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:03:10.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.441+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:10.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:10.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:10.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:10.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:10.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:10.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:10.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:10.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.485+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:10.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.797+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:10.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:10.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:10.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:10.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:10.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:10.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.849+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:10.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.849+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:10.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:10.849+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:12.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:12.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:12.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:12.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:12.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:12.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.949+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:12.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.949+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:12.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:12.953+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:14.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.469+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:14.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:14.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:14.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:14.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:14.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.513+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:14.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.513+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:14.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:14.513+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:14.849 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:14.845+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:03:14.872 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:14.861+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:14.872 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:14.861+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:14.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:14.861+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:16.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.721+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:16.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:16.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:16.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:16.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:16.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:16.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.753+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:16.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.753+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:16.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:16.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:16.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:16.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:16.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:16.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.901+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:16.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.901+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:16.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:16.905+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:17.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:17.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:17.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:17.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:17.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:17.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:17.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:17.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:17.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:17.429+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:17.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:17.429+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:17.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:17.429+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:18.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:18.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:18.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:18.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:18.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:18.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.693+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:18.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.693+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:18.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.713+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:18.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:18.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:18.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:18.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:18.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.817+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:18.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:18.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:18.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:18.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:18.921+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:19.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.317+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:19.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:19.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:19.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:19.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:19.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:19.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:19.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:19.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:19.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:19.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:19.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:19.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:19.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.745+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:19.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.745+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:19.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.753+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:19.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.789+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:19.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:19.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:19.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:19.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:19.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.813+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:19.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:19.813+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:20.267 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:03:20.268 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:03:20.268 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:03:20.292 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:03:21.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.305+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:21.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:21.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:21.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:21.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:21.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:21.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.353+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:21.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.353+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:21.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.353+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:21.456 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3703893601 2023-10-14T00:03:21.456 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:03:21.457 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241801 2023-10-14T00:03:21.457 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:03:21.457 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3703893601 ]] 2023-10-14T00:03:21.458 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3703893601 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\0\3\8\9\3\6\0\1 ]] 2023-10-14T00:03:21.458 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:03:21.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:21.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:21.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:21.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.989+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:22.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:21.997+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:24.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.193+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:24.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:24.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:24.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:24.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:24.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:24.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.269+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:24.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.269+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:24.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:24.269+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:26.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.365+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:26.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:26.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:26.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:26.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:26.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.425+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:26.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.425+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:26.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.425+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:26.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:26.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:26.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:26.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:26.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:26.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.521+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:26.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.521+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:26.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.521+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:26.544 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:26.541+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:26.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:26.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:26.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:26.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:26.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.701+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:26.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.701+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:26.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.713+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:26.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.777+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:03:26.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.777+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:03:26.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.797+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:26.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:26.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:26.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:26.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:26.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:26.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:26.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.833+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:26.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:26.849+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:27.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:27.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:27.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:27.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:27.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:27.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:27.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:27.531 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:27.524+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:27.537 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:27.532+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:27.564 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:27.560+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:27.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:27.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:27.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:27.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:27.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.724+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:27.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.724+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:27.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:27.724+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:29.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:29.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:29.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:29.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:29.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:29.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:29.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:29.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:29.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:29.208+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:29.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:29.208+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:29.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:29.228+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:31.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:31.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:31.459 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:03:31.459 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:03:31.461 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:03:31.464 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:03:33.071 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3703893601 2023-10-14T00:03:33.072 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:03:33.072 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241813 2023-10-14T00:03:33.072 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:03:33.073 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3703893601 ]] 2023-10-14T00:03:33.073 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3703893601 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\0\3\8\9\3\6\0\1 ]] 2023-10-14T00:03:33.073 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:03:34.039 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:34.032+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:34.044 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:34.040+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:34.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:34.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:34.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.112+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:34.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.192+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:34.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:34.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:34.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:34.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:34.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:34.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:34.236+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:35.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:35.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:35.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:35.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:35.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:35.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:35.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:35.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:35.384+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:37.093 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:37.088+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:37.232 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:37.228+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:03:37.238 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:37.232+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:38.976 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:38.972+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:39.715 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:39.708+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:03:39.741 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:39.736+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:39.754 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:39.748+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:40.964 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:40.956+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:41.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.024+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:41.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:41.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:41.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:41.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:41.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.072+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:41.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.072+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:41.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.072+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:41.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.440+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:41.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:41.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:41.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:41.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:41.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:41.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:41.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:41.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:42.242 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:42.236+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:42.305 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:42.300+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:42.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.504+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:42.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:42.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:42.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:42.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.576+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:42.787 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:42.784+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:42.798 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:42.792+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:42.799 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:42.792+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:42.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:42.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:42.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:42.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.952+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:42.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.952+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:42.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:42.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:43.079 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:03:43.080 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:03:43.081 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:03:43.126 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:03:43.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.748+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:43.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:43.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:43.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:43.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:43.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.804+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:43.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.804+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:43.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:43.808+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:44.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:44.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:44.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:44.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:44.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:44.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:44.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:44.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:44.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:44.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:44.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:44.340+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:44.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:44.360+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:44.536 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3703893601 2023-10-14T00:03:44.536 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:03:44.537 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241824 2023-10-14T00:03:44.537 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:03:44.537 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3703893601 ]] 2023-10-14T00:03:44.538 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3703893601 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\0\3\8\9\3\6\0\1 ]] 2023-10-14T00:03:44.538 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:03:45.853 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:45.844+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:46.012 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:46.008+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:46.023 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:46.020+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:46.033 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:46.028+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:46.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.244+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:46.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:46.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:46.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:46.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:46.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:46.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.292+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:46.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.292+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:46.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.296+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:46.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.800+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:46.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:46.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:46.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:46.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:46.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:46.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.860+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:46.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.860+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:46.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:46.860+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:47.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.460+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:47.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:47.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:47.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:47.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:47.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:47.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.508+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:47.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.508+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:47.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:47.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:48.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:48.616+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:48.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:48.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:48.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:48.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:48.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:48.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:48.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:48.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:48.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:48.644+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:49.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.600+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:49.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:49.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:49.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:49.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:49.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.624+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:49.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:49.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:49.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:49.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:50.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:50.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:50.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:50.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:50.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:50.024+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:50.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:50.024+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:50.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:50.040+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:50.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:50.864+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:51.913 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:51.908+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:51.913 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:51.908+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:54.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:54.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:54.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:54.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:54.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.048+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:54.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.048+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:54.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.092+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:54.539 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:03:54.539 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:03:54.544 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:03:54.546 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:03:54.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.564+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:03:54.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:54.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:54.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:54.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:54.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:54.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.672+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:54.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.672+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:54.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:54.700+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:55.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:55.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:55.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:55.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:55.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:55.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.956+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:55.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.956+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:55.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:55.964+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:56.106 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3703893601 2023-10-14T00:03:56.106 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:03:56.107 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241836 2023-10-14T00:03:56.107 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:03:56.107 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3703893601 ]] 2023-10-14T00:03:56.108 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3703893601 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\0\3\8\9\3\6\0\1 ]] 2023-10-14T00:03:56.108 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:03:56.731 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:56.724+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:03:56.770 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:56.764+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:03:56.793 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:56.788+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:57.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:57.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:57.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:57.744+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:58.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:58.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:58.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:58.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:58.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.376+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:58.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:58.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:58.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:58.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:58.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:58.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.680+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:58.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.680+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:03:58.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:58.680+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:03:59.164 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:59.156+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:59.322 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:03:59.316+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:03:59.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:03:59.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:03:59.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:03:59.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:03:59.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:03:59.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.980+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:03:59.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.980+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:00.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:03:59.996+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:01.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.076+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:01.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:01.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:01.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:01.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:01.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:01.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.100+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:01.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:01.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:01.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:01.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:01.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.264+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:01.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.264+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:01.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:01.268+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:01.789 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:01.784+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:04:01.797 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:01.792+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:02.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:02.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:02.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:02.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:02.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:02.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.936+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:02.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.936+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:02.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:02.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:04.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:04.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:04.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:04.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:04.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:04.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:04.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:04.504+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:04.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:04.504+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:04.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:04.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:05.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:05.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:05.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.556+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:05.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.588+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:04:05.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:05.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:05.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:05.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.656+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:05.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:05.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:05.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:05.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:05.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.904+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:05.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.912+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:05.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:05.912+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:06.109 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:04:06.111 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:04:06.117 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:04:06.131 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:04:07.474 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:07.468+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:07.765 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529501161 2023-10-14T00:04:07.765 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:04:07.766 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241847 2023-10-14T00:04:07.767 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:04:07.767 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529501161 ]] 2023-10-14T00:04:07.767 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529501161 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\0\3\8\9\3\6\0\1 ]] 2023-10-14T00:04:07.767 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/1529501161 2023-10-14T00:04:08.248 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:08.244+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:08.262 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:08.256+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:08.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:08.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:08.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:08.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:08.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:08.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:08.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:08.416+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:08.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:08.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:08.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:08.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:08.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:08.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:09.441 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:09.432+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:09.473 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:09.468+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:09.942 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:09.936+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:10.673 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/1529501161 until 2023-10-14T01:04:09.915115+0000 (3600 sec) 2023-10-14T00:04:10.741 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/1529501161 2023-10-14T00:04:10.742 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:04:10.775 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:10.768+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:11.174 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:11.168+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:11.191 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:11.188+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:12.048 INFO:tasks.workunit.client.0.smithi183.stderr:listed 4 entries 2023-10-14T00:04:12.120 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/1529501161 2023-10-14T00:04:12.120 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:04:12.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:12.404+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:12.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:12.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:12.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:12.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:12.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:12.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:13.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.000+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:04:13.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.000+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:04:13.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:13.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:13.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:13.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:13.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:13.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:13.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:13.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:13.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:13.908+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:14.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:14.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:14.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:14.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:14.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:14.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:15.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:15.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:15.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:15.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:15.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:15.004+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:15.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:15.004+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:15.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:15.024+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:17.005 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:17.004+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:17.018 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:17.016+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:17.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.248+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:04:17.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.248+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:04:17.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.256+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:17.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:17.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:17.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:17.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:17.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:17.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:17.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:17.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:17.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.712+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:17.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:17.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:17.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:17.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:17.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:17.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.760+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:17.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.760+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:17.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:17.760+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:19.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:19.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:19.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:19.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:19.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:19.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.356+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:19.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.356+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:19.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:19.356+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:20.581 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:20.579+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:20.591 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:20.587+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:20.592 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:20.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:21.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:21.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:21.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:21.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:22.130 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:04:22.131 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:04:22.145 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:04:22.148 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:04:22.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.363+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:22.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:22.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:22.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:22.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:22.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:22.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.391+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:22.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.391+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:22.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.403+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:22.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.523+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:22.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:22.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:22.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:22.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:22.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:22.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.595+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:22.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.595+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:22.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:22.595+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:23.576 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529501161 2023-10-14T00:04:23.579 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:04:23.580 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241863 2023-10-14T00:04:23.581 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:04:23.581 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529501161 ]] 2023-10-14T00:04:23.581 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529501161 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\0\1\1\6\1 ]] 2023-10-14T00:04:23.581 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:04:23.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:23.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:23.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:23.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:23.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:23.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:23.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:23.735+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:23.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:23.735+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:23.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:23.755+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:23.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:23.887+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:23.899 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:23.899+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:24.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.371+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:24.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:24.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:24.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:24.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:24.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:24.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.443+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:24.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.443+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:24.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.479+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:24.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.487+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:04:24.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.487+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:04:24.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.491+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:24.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:24.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:24.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:24.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:24.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:24.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.535+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:24.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.535+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:24.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:24.535+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:24.750 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:24.747+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:24.785 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:24.783+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:25.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:25.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:25.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:25.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:25.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:25.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.327+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:25.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.327+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:25.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.327+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:25.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:25.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:25.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:25.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:25.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:25.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.703+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:25.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.703+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:25.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:25.711+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:29.343 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:29.339+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:29.357 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:29.355+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:30.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:30.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:30.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:30.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:30.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:30.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.135+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:30.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.143+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:30.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:30.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:30.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:30.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:30.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.767+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:30.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.767+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:30.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:30.783+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:32.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:32.267+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:32.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:32.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:32.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:32.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:32.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:32.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:32.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:32.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:32.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:32.299+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:32.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:32.303+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:32.450 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:04:32.447+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:04:32.450 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:04:32.447+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:04:33.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.159+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:33.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:33.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:33.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:33.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:33.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:33.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.203+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:33.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.203+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:33.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:33.203+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:33.585 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:04:33.588 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:04:33.598 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:04:33.609 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:04:34.070 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:34.067+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:34.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529501161 2023-10-14T00:04:34.656 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:04:34.657 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241874 2023-10-14T00:04:34.657 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:04:34.657 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529501161 ]] 2023-10-14T00:04:34.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529501161 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\0\1\1\6\1 ]] 2023-10-14T00:04:34.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:04:34.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:34.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:34.679 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:34.679+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:34.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:34.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:34.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:34.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:34.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:34.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:34.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:34.703+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:34.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:34.703+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:34.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:34.711+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:35.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:35.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:35.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:35.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:35.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:35.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.939+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:35.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.939+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:35.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:35.943+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:36.467 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:36.463+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:36.474 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:36.471+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:36.631 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:36.631+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:36.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:36.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:36.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:36.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:36.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:36.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:36.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:36.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:36.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:36.787+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:37.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:37.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:37.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:37.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:37.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:37.263+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:37.335 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:37.335+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:37.347 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:37.347+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:40.512 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:40.511+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:40.547 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:40.535+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:41.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.043+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:41.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:41.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:41.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:41.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:41.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:41.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.071+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:41.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.071+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:41.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.099+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:41.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.471+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:41.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:41.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:41.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.515+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:41.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:41.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:41.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.543+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:41.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.543+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:41.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:41.543+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:42.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.159+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:42.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:42.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:42.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:42.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:42.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.191+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:42.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.191+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:42.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:42.191+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:42.199 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:42.199+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:44.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:44.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:44.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:44.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:44.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:44.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.503+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:44.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.503+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:44.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.511+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:44.574 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:44.571+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:44.597 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:44.595+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:44.609 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:44.607+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:44.667 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:04:44.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.671+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:04:44.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.671+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:04:44.675 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:04:44.680 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:04:44.683 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:04:44.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.711+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:44.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:44.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:44.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:44.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:44.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:44.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:44.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:44.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:44.779+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:45.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:45.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:45.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:45.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:45.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:45.575+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:45.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:45.615+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:45.982 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529501161 2023-10-14T00:04:45.983 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:04:45.984 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241885 2023-10-14T00:04:45.984 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:04:45.984 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529501161 ]] 2023-10-14T00:04:45.985 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529501161 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\0\1\1\6\1 ]] 2023-10-14T00:04:45.985 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:04:47.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:47.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:47.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:47.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:47.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:47.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.411+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:47.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.411+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:47.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:47.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:47.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:47.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:47.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.675+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:47.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.675+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:47.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:47.675+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:49.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:49.311+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:49.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:49.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:49.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:49.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:49.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:49.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:49.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:49.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:49.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:49.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:49.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:49.351+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:51.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:51.115+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:51.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:51.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:51.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:51.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:51.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:51.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:51.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:51.167+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:52.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:52.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:52.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:52.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:52.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:52.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:52.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:52.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:52.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:52.731+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:52.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:52.735+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:53.017 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:53.015+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:54.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:54.451+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:54.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:54.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:54.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:54.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:54.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:54.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:54.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:54.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:54.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:54.487+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:54.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:54.487+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:55.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:55.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:55.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:55.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.571+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:55.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.571+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:55.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.583+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:55.612 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:55.611+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:55.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.619+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:55.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:55.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:55.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:55.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:55.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:55.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.699+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:55.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.699+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:55.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:55.699+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:55.987 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:04:55.987 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:04:55.987 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:04:56.022 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:04:56.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:56.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:56.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:56.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:56.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:56.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.639+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:56.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.639+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:56.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.639+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:56.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:56.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:56.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:56.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:56.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.799+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:56.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.799+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:56.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.807+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:56.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.831+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:04:56.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:56.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:56.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:56.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:56.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:56.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.887+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:56.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.887+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:56.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:56.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:57.264 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529501161 2023-10-14T00:04:57.264 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:04:57.265 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241897 2023-10-14T00:04:57.265 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:04:57.265 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529501161 ]] 2023-10-14T00:04:57.266 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529501161 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\0\1\1\6\1 ]] 2023-10-14T00:04:57.266 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:04:58.015 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:58.011+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:04:58.024 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:58.023+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:04:58.025 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:04:58.023+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:04:58.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.383+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:04:58.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:58.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:58.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:58.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:58.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.423+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:58.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.423+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:58.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.423+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:58.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:04:58.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:04:58.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:04:58.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:04:58.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:04:58.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.895+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:04:58.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.895+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:04:58.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:58.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:04:59.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:04:59.799+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.031+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.031+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.031+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.031+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.031+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.034 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.031+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.038 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.035+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.038 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.035+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.043+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.043+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1eb00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.091 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.087+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.091 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.087+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1ed00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.091+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.091+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.091+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2931b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.091+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb1e1b980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.095+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.099+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.099+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.099+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.099+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1f000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.099+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb295d700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.099+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.099+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c3af00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.107+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.107+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.107+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.107+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.113 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.111+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.113 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.111+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1e600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.111+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.111+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1f200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.119+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.119+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1f400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.123+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.123+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1f600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.128 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.127+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.128 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.127+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1f800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.131+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:05:00.131+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c1fa00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:05:00.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:00.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:00.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:00.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:00.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.439+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:00.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.439+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:00.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:00.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:00.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:00.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:00.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:00.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.882+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:00.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.882+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:00.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:00.890+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:01.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.126+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:01.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:01.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:01.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:01.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:01.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.190+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:01.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.190+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:01.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.190+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:01.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.486+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:01.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:01.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:01.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:01.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:01.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:01.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.542+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:01.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.542+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:01.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.558+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:01.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:01.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:01.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:01.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:01.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:01.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.818+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:01.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.818+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:01.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:01.838+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:02.327 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:02.326+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:02.327 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:02.326+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:02.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:02.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:02.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:02.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:02.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:02.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.586+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:02.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.586+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:02.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:02.586+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:03.403 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:03.402+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:03.432 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:03.430+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:04.881 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:04.878+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:04.899 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:04.898+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:05.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:05.954+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:06.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:06.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:06.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:06.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:06.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:06.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.506+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:06.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.534+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:06.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.538+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:06.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.946+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:06.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:06.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:06.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:06.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:06.996 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:06.994+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:07.007 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:07.006+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:07.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:07.034+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:07.267 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:05:07.268 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:05:07.268 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:05:07.271 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:05:08.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:08.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:08.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:08.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.558+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:08.711 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529501161 2023-10-14T00:05:08.714 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:05:08.728 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241908 2023-10-14T00:05:08.728 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:05:08.729 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529501161 ]] 2023-10-14T00:05:08.729 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529501161 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\0\1\1\6\1 ]] 2023-10-14T00:05:08.729 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:05:08.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.962+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:08.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:08.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:08.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:08.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:08.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:08.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.990+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:08.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.990+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:08.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:08.990+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:09.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:09.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:09.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:09.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:09.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:09.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.242+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:09.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.242+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:09.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:09.250+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:10.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:10.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:10.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:10.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:10.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:10.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.190+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:10.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.190+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:10.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.190+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:10.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:10.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:10.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:10.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:10.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:10.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.522+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:10.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.522+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:10.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.530+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:10.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:10.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:10.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:10.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:10.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:10.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:10.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:10.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.678+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:10.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:10.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:10.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:10.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:10.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:10.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:10.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:10.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:10.918+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:11.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:11.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:11.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:11.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:11.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:11.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.410+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:11.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.410+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:11.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:11.426+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:12.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:12.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:12.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:12.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:12.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:12.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:12.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:12.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:12.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:12.138+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:12.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:12.138+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:12.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:12.138+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:14.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:14.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:14.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:14.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:14.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.322+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:14.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.322+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:14.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.322+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:14.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:14.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:14.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:14.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:14.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.838+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:14.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:14.846+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:16.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.118+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:16.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:16.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:16.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:16.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:16.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.146+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:16.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.146+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:16.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.150+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:16.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:16.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:16.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:16.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:16.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.690+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:16.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.690+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:16.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:16.706+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:17.899 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:17.894+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:17.914 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:17.910+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:18.316 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:18.314+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:18.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:18.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:18.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.554+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:18.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.646+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:18.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:18.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:18.730 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:05:18.738 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:05:18.743 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:05:18.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:18.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:18.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.746+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:18.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.746+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:18.753 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:05:18.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.814+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:18.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:18.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:18.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:19.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:19.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:19.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:19.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:19.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:19.006+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:19.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:19.006+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:19.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:19.006+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:19.332 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:19.330+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:05:19.350 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:19.346+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:19.354 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:19.350+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:19.358 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:19.354+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:20.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1235739348 2023-10-14T00:05:20.124 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:05:20.126 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241920 2023-10-14T00:05:20.126 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:05:20.126 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1235739348 ]] 2023-10-14T00:05:20.126 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1235739348 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\0\1\1\6\1 ]] 2023-10-14T00:05:20.127 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/1235739348 2023-10-14T00:05:20.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:20.422+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:21.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.242+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:21.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:21.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:21.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:21.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:21.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:21.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.254+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:21.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.254+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:21.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:21.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:22.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:22.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:22.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:22.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:22.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:22.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:22.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:22.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:22.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:22.422+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:22.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:22.422+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:22.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:22.430+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:22.735 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/1235739348 until 2023-10-14T01:05:22.085224+0000 (3600 sec) 2023-10-14T00:05:22.782 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/1235739348 2023-10-14T00:05:22.784 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:05:22.967 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:22.966+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:05:22.981 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:22.978+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:22.994 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:22.990+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:23.000 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:22.998+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:24.106 INFO:tasks.workunit.client.0.smithi183.stderr:listed 5 entries 2023-10-14T00:05:24.160 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/1235739348 2023-10-14T00:05:24.161 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:05:24.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:24.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:24.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:24.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.478+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:24.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.478+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:24.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.482+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:24.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:24.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:24.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:24.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:24.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.662+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:24.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.670+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:24.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.670+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:24.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:24.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:24.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:24.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:24.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:24.686+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:25.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:25.302+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:25.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:25.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:25.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:25.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:25.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:25.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:25.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:25.330+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:25.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:25.330+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:25.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:27.622 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:27.618+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:27.645 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:27.642+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:27.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:27.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:27.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:27.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:27.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:27.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:27.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:27.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:27.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:27.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:27.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:27.990+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:27.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:27.990+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:31.395 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:31.386+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:31.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:31.398+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:31.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:31.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:31.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:31.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:31.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:31.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:31.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:31.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:31.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:31.710+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:31.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:31.710+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:31.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:31.710+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:32.542 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:32.538+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:33.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:33.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:33.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:33.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:33.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:33.646+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:34.174 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:05:34.175 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:05:34.176 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:05:34.183 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:05:35.494 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1235739348 2023-10-14T00:05:35.494 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:05:35.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241935 2023-10-14T00:05:35.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:05:35.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1235739348 ]] 2023-10-14T00:05:35.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1235739348 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\2\3\5\7\3\9\3\4\8 ]] 2023-10-14T00:05:35.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:05:36.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.462+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:05:36.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.502+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:36.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:36.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:36.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.522+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:36.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:36.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:36.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.526+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:36.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.526+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:36.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:36.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:36.631 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:36.626+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:37.370 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:37.366+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:37.402 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:37.394+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:38.204 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:38.201+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:38.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:38.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:38.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.637+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:38.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:38.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:38.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:38.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:38.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:38.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:38.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.977+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:38.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.977+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:38.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:38.977+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:39.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:39.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:39.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:39.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:39.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:39.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:39.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:39.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:39.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:39.597+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:39.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:39.597+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:39.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:39.597+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:39.683 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:39.677+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:39.719 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:39.713+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:40.099 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:40.093+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:40.106 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:40.101+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:40.106 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:40.101+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:40.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:40.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:40.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:40.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:40.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:40.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:40.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:40.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:40.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:40.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.421+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:40.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:40.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:40.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:40.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:40.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.713+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:40.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:40.713+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:41.579 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:41.573+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:42.099 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:42.093+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:42.230 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:42.225+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:42.236 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:42.233+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:44.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:44.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:44.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:44.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.109+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:44.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.113+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:44.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:44.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:44.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:44.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.981+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:44.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:44.989+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:45.500 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:05:45.500 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:05:45.508 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:05:45.535 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:05:45.647 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:45.645+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:45.675 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:45.673+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:45.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:45.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:45.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:45.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:45.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:45.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:45.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:45.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:45.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:45.801+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:45.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:45.801+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:45.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:45.801+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:46.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:46.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:46.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:46.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:46.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:46.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.493+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:46.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.493+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:46.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.509+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:46.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:46.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:46.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:46.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:46.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:46.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.577+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:46.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.577+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:46.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.581+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:46.806 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1235739348 2023-10-14T00:05:46.819 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:05:46.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241946 2023-10-14T00:05:46.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:05:46.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1235739348 ]] 2023-10-14T00:05:46.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1235739348 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\2\3\5\7\3\9\3\4\8 ]] 2023-10-14T00:05:46.822 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:05:46.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:46.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:46.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:46.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:46.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:46.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.901+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:46.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:46.905+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:47.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.285+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:47.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:47.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:47.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:47.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:47.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.349+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:47.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.373+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:05:47.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.373+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:05:47.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.377+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:47.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:47.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:47.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:47.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:47.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:47.421+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:48.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.401+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:48.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:48.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:48.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:48.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:48.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:48.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.453+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:48.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.453+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:48.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.473+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:48.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:48.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:48.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:48.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:48.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.677+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:48.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.677+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:48.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:48.677+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:48.719 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:48.717+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:48.739 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:48.737+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:49.977 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:49.969+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:50.104 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:50.101+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:51.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:51.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:51.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:51.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:51.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.301+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:51.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:51.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:51.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:51.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:51.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:51.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.477+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:51.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.477+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:51.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:51.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:52.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:52.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:52.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:52.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:52.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:52.669+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:53.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:53.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:53.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:53.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:53.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:53.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:53.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:53.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:53.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:53.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:53.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:53.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:54.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:54.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:54.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:54.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:54.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.373+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:05:54.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.381+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:54.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:54.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:54.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:54.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:54.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:54.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:54.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:54.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:54.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:54.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.733+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:54.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.733+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:54.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:54.733+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:55.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:55.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:55.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:55.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:55.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.701+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:55.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:55.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:55.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:55.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.749+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:56.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:55.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:56.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:56.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:56.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:56.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:56.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:56.013+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:56.504 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:56.501+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:05:56.508 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:56.505+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:05:56.521 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:05:56.517+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:05:56.831 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:05:56.831 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:05:56.831 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:05:56.857 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:05:57.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:57.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:57.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:57.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:57.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:57.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:57.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:57.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:57.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.213+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:57.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.549+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:57.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:57.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.585+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.637+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:57.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.637+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:05:57.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:57.661+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:58.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.197+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:05:58.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:58.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:58.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:58.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:58.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:58.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:05:58.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:58.217+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:05:58.249 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1235739348 2023-10-14T00:05:58.249 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:05:58.250 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241958 2023-10-14T00:05:58.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:05:58.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1235739348 ]] 2023-10-14T00:05:58.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1235739348 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\2\3\5\7\3\9\3\4\8 ]] 2023-10-14T00:05:58.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:05:59.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:59.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:05:59.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:59.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:05:59.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:59.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:05:59.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:05:59.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:05:59.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:05:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.009+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.014 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.009+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.014 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.009+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.014 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.009+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.019 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.013+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.019 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.013+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.021+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.021+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.029+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.029+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.053+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.053+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.065+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.065+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.065+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.065+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.065+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.065+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.073+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.073+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.089+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.089+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.113+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.113+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.129+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.129+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e71f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.149 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.145+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.150 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.145+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d28100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.150 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.145+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.150 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.145+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d28300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.157 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.153+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.157 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.153+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d28500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.191 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.189+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.191 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.189+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d28700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.192 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.189+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.192 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.189+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d28900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.199 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.197+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.199 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.197+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d28b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.200 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.197+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:00.200 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:06:00.197+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d28d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:06:02.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:02.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:02.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:02.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:02.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:02.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:02.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:02.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:03.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:03.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:03.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:03.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:03.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:03.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.297+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:03.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.297+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:03.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:03.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:04.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:04.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:04.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:04.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:04.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:04.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.765+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:04.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.765+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:04.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:04.797+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:05.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:05.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:05.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:05.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:05.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:05.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.049+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:05.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.049+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:05.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:05.061+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:05.881 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:05.877+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:05.889 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:05.885+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:06.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:06.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:06.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:06.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:06.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:06.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.521+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:06.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.529+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:06.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:06.533+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:07.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:07.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:07.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:07.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:07.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:07.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:07.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:07.717+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:07.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:07.717+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:07.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:07.717+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:08.252 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:06:08.253 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:06:08.253 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:06:08.259 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:06:09.258 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:09.253+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:09.774 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1235739348 2023-10-14T00:06:09.774 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:06:09.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241969 2023-10-14T00:06:09.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:06:09.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1235739348 ]] 2023-10-14T00:06:09.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1235739348 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\2\3\5\7\3\9\3\4\8 ]] 2023-10-14T00:06:09.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:06:10.102 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:10.097+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:10.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:10.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:10.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:10.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:10.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:10.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:10.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:10.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:10.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:10.605+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:10.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:10.605+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:10.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:10.609+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:11.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:11.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:11.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:11.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:11.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:11.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:11.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:11.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:11.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:11.113+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:11.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:11.113+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:11.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:11.125+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:11.404 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:11.401+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:12.443 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:12.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:13.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.509+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:06:13.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.509+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:06:13.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.521+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:13.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:13.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:13.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:13.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:13.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:13.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.529+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:13.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.529+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:13.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:13.529+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:15.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:15.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:15.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:15.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:15.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.221+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:15.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.221+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:15.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:15.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:15.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:15.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:15.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:15.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:15.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:15.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:15.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.809+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:06:15.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.845+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:15.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:15.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:15.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:15.907 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:15.905+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:15.926 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:15.921+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:15.926 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:15.921+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:15.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:15.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:15.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.933+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:15.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.933+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:15.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:15.953+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:16.105 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:16.101+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:16.111 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:16.109+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:16.126 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:16.125+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:16.254 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:16.249+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:16.281 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:16.277+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:16.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:16.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:16.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:16.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:17.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.893+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:17.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:17.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:17.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:17.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:17.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:17.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:17.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.953+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:17.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:17.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:18.434 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:18.429+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:06:18.435 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:18.433+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:18.464 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:18.461+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:18.480 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:18.473+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:19.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.177+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:19.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:19.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:19.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:19.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:19.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:19.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.205+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:19.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.205+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:19.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:19.233+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:19.654 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:19.653+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:19.662 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:19.657+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:19.662 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:19.657+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:19.791 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:06:19.792 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:19.789+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:19.794 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:06:19.795 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:19.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:19.795 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:19.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:19.800 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:06:19.818 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:06:20.667 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:20.665+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:20.679 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:20.677+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:20.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.929+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:20.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:20.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:20.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:20.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:20.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:20.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.973+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:20.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.973+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:20.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:20.973+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:21.012 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:21.009+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:21.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:21.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:21.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:21.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:21.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:21.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:21.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:21.225+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:21.243 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1235739348 2023-10-14T00:06:21.249 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:06:21.258 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241981 2023-10-14T00:06:21.259 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:06:21.259 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1235739348 ]] 2023-10-14T00:06:21.259 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1235739348 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\2\3\5\7\3\9\3\4\8 ]] 2023-10-14T00:06:21.260 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:06:21.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:21.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:06:21.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:21.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:21.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:21.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:21.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:21.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:22.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:22.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:22.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:22.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:22.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.425+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:22.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:22.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:22.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:22.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:22.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:22.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:22.585+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:23.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.717+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:23.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:23.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:23.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:23.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:23.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:23.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:23.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:23.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:23.829+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:25.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:25.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:25.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:25.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:25.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:25.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:25.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:25.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:25.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.189+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:25.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.189+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:25.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:25.197+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:26.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:26.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:26.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:26.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:26.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:26.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.581+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:26.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.981+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:26.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:26.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:26.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:26.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:27.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:27.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:27.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.009+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:27.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.033+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:27.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.033+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:27.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.261+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:27.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:27.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:27.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:27.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:27.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:27.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.301+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:27.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.301+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:27.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:27.329+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:28.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:28.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:28.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:28.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:28.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:28.905+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:30.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:30.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:30.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:30.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:30.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:30.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:30.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:30.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:30.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:30.317+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:30.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:30.317+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:30.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:30.325+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:30.648 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:30.645+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:31.260 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:06:31.261 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:06:31.262 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:06:31.267 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:06:31.738 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:31.733+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:31.744 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:31.741+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:31.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.853+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:31.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:31.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:31.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:31.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.941+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:31.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.941+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:31.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.941+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:31.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.941+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:31.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:31.941+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:32.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1555743736 2023-10-14T00:06:32.742 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:06:32.744 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697241992 2023-10-14T00:06:32.744 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:06:32.745 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1555743736 ]] 2023-10-14T00:06:32.745 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1555743736 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\2\3\5\7\3\9\3\4\8 ]] 2023-10-14T00:06:32.745 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/1555743736 2023-10-14T00:06:34.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:34.125+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:34.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:34.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:34.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:34.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:34.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:34.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:34.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:34.177+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:34.763 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/1555743736 until 2023-10-14T01:06:34.115113+0000 (3600 sec) 2023-10-14T00:06:34.841 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:06:34.842 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/1555743736 2023-10-14T00:06:34.918 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:34.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:35.933 INFO:tasks.workunit.client.0.smithi183.stderr:listed 6 entries 2023-10-14T00:06:35.960 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/1555743736 2023-10-14T00:06:35.960 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:06:36.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:36.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:36.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:36.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:36.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.021+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:36.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.021+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:36.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.021+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:36.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:36.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:36.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:36.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:36.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:36.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.409+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:36.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.409+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:36.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:36.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:38.597 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:38.593+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:38.598 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:38.593+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:39.270 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:39.265+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:40.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:40.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:40.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:40.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:40.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:40.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:40.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:40.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:40.045+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:41.091 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:41.089+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:41.112 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:41.109+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:41.138 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:41.133+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:06:41.140 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:41.137+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:41.155 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:41.153+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:41.156 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:41.153+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:42.949 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:42.945+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:43.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:43.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:43.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:43.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:43.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:43.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:43.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:43.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:43.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:43.901+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:43.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:43.901+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:43.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:43.909+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:44.304 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:44.301+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:06:44.310 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:44.309+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:44.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:44.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:44.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:44.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:44.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:44.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:44.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:44.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:44.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:44.845+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:44.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:44.845+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:44.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:44.845+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:45.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.657+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:45.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:45.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:45.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:45.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:45.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:45.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:45.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:45.967 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:06:45.967 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:06:45.969 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:06:45.994 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:06:46.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.209+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:46.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:46.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:46.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:46.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:46.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:46.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.309+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:46.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.309+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:46.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:46.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:47.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:47.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:47.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:47.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:47.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:47.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.117+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:47.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.117+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:47.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.137+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:47.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.257+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:47.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:47.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:47.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:47.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:47.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:47.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.329+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:47.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.329+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:47.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:47.341+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:47.434 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1555743736 2023-10-14T00:06:47.435 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:06:47.435 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242007 2023-10-14T00:06:47.435 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:06:47.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1555743736 ]] 2023-10-14T00:06:47.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1555743736 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\5\5\7\4\3\7\3\6 ]] 2023-10-14T00:06:47.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:06:48.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:48.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:48.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:48.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:48.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:48.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.169+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:48.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.169+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:48.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.169+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:48.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.885+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:48.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:48.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:48.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:48.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:48.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:48.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.917+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:48.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.917+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:48.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:48.917+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:50.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:50.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:50.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:50.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:50.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:50.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.949+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:50.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.949+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:50.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:50.953+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:51.614 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:51.613+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:51.664 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:51.661+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:51.674 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:51.673+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:52.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.321+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:52.357 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:52.353+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:52.360 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:52.357+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:52.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:52.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:52.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:52.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:52.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.369+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:52.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.369+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:52.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.369+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:52.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.649+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:06:52.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.669+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:52.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:52.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:52.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:52.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:52.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:52.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.733+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:52.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.733+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:52.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:52.741+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:53.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:53.789+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:53.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:53.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:53.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:53.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:53.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:53.857+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:53.882 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:53.881+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:55.230 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:55.229+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:55.262 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:55.261+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:55.564 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:55.561+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:55.566 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:55.565+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:06:55.566 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:55.565+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:55.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:55.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:55.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:55.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:55.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:55.733+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:56.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.329+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:56.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:56.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:56.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:56.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:56.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:56.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:56.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:56.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:57.440 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:06:57.445 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:06:57.445 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:06:57.446 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:06:58.228 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:58.225+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:06:58.248 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:06:58.245+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:06:58.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:58.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.329+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:58.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.329+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:58.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.349+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:58.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.485+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:06:58.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:06:58.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:06:58.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:06:58.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:06:58.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:06:58.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.525+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:06:58.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.525+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:06:58.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:06:58.529+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:06:59.042 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1555743736 2023-10-14T00:06:59.042 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:06:59.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242019 2023-10-14T00:06:59.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:06:59.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1555743736 ]] 2023-10-14T00:06:59.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1555743736 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\5\5\7\4\3\7\3\6 ]] 2023-10-14T00:06:59.045 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:07:00.018 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.013+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.018 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.013+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb25d4f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.029+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.029+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d29380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.033+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.033+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d29180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.045 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.041+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.045 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.041+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d28f80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.077+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.077+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.077+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d28c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.077+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffa080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.101+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.101+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffa680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.113+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffa500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffac00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffa880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.121 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffa300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.121 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffaa80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.121 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffae80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.121 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffb080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.122 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.121+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.123 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.121+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffb280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.125+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.125+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffb480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.125+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.125+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffb680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.129+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.129+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffb880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.129+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.129+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffba80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.146 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.137+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.147 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:07:00.137+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffbc80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:07:00.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:00.181+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:00.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:00.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:00.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:00.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:00.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:00.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:00.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:00.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:00.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:00.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:00.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:00.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:00.309 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:00.305+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:01.292 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:01.289+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:02.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:02.317+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:02.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:02.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:02.366 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:02.361+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:02.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:02.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:02.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.405+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:02.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.677+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:02.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:02.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:02.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:02.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:02.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:02.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:02.737+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:03.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.441+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:03.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:03.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:03.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:03.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:03.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.481+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:03.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.481+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:03.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.489+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:03.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:03.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:03.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:03.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.921+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:03.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.921+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:03.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:03.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:09.059 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:07:09.066 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:07:09.070 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:07:09.077 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:07:10.223 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:10.220+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:07:10.228 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:10.224+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:10.229 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:10.224+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:10.570 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1555743736 2023-10-14T00:07:10.570 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:07:10.571 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242030 2023-10-14T00:07:10.571 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:07:10.571 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1555743736 ]] 2023-10-14T00:07:10.572 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1555743736 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\5\5\7\4\3\7\3\6 ]] 2023-10-14T00:07:10.572 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:07:11.863 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:11.860+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:07:11.880 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:11.876+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:07:11.896 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:11.892+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:11.910 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:11.908+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:12.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:12.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:12.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:12.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:12.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:12.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:12.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:12.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:12.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:12.148+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:12.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:12.148+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:14.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:14.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:14.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:14.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:14.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:14.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:15.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:15.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:15.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:15.132+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:17.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:17.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:17.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:17.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.096+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:17.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.104+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:17.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:17.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:17.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:17.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:17.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:17.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:17.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:17.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.732+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:17.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.732+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:17.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:17.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:17.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:17.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:17.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:17.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.900+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:17.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.900+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:17.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.928+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:17.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.956+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:17.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:17.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:17.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:17.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:17.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:17.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:17.976+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:18.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.000+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:18.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.240+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:18.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:18.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:18.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:18.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:18.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:18.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:18.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:18.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:18.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:18.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:18.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:18.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:18.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:19.318 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:19.316+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:19.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.604+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:19.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.620+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:19.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:19.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:19.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:19.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:19.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.664+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:19.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.844+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:19.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:19.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:19.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:19.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:19.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:19.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.888+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:19.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.888+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:19.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:19.888+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:20.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:20.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:20.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:20.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:20.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:20.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:20.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.272+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:20.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:20.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:20.580 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:07:20.609 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:07:20.610 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:07:20.613 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:07:20.786 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:07:20.784+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:07:22.128 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1555743736 2023-10-14T00:07:22.129 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:07:22.130 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242042 2023-10-14T00:07:22.130 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:07:22.130 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1555743736 ]] 2023-10-14T00:07:22.130 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1555743736 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\5\5\7\4\3\7\3\6 ]] 2023-10-14T00:07:22.130 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:07:22.590 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:22.588+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:07:22.623 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:22.620+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:22.634 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:22.632+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:23.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:23.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:23.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:23.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:23.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:23.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:23.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:23.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:23.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:23.964+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:23.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:23.964+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:23.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:23.964+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:24.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:24.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:24.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:24.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:24.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:24.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.252+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:24.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.252+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:24.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:24.252+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:25.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:25.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:25.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:25.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:25.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:25.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:25.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:25.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:25.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:25.164+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:25.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:25.164+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:25.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:25.184+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:25.513 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:25.508+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:07:25.523 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:25.520+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:25.541 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:25.536+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:26.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.608+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:26.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:26.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:26.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:26.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:26.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:26.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.668+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:26.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.668+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:26.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:26.700+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:26.730 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:26.724+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:27.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:27.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:27.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:27.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:27.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.084+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:27.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.084+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:27.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.108+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:27.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.360+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:27.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:27.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:27.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:27.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:27.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:27.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:27.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:27.412+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:27.876 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:27.864+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:07:27.897 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:27.896+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:27.913 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:27.908+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:28.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:28.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:28.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:28.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:28.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:28.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.892+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:28.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.892+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:28.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:28.920+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:29.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:29.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:29.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:29.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:29.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:29.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:29.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:29.192+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:29.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:29.192+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:29.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:29.192+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:30.578 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:30.576+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:07:30.647 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:30.644+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:30.765 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:30.760+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:30.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.896+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:30.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:30.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:30.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:30.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:30.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:30.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:30.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:30.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:30.908+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:31.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:31.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:31.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:31.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:31.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.484+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:31.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.484+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:31.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.512+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:31.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:31.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:31.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:31.964+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:32.144 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:07:32.148 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:07:32.152 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:07:32.164 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:07:33.493 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1555743736 2023-10-14T00:07:33.496 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:07:33.499 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242053 2023-10-14T00:07:33.499 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:07:33.499 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1555743736 ]] 2023-10-14T00:07:33.499 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1555743736 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\5\5\7\4\3\7\3\6 ]] 2023-10-14T00:07:33.500 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:07:34.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.260+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:34.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:34.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:34.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:34.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:34.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:34.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:34.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:34.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.304+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:34.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:34.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:34.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:34.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.588+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:34.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:34.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:34.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:34.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.628+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:34.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.628+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:34.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:34.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:34.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.652+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:34.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.940+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:34.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:34.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:34.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:35.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:35.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:35.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:35.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:35.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:35.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:35.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:35.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:35.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:35.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:35.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:35.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:35.313 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:35.308+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:07:35.316 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:35.312+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:07:35.322 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:35.320+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:35.328 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:35.324+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:37.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:37.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:37.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:37.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:37.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:37.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.332+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:37.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.332+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:37.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:37.336+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:38.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:38.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:38.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:38.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.104+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:38.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.104+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:38.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.112+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:38.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.536+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:38.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.536+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:07:38.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.548+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:38.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:38.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:38.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:38.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:38.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:38.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.576+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:38.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.576+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:38.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.576+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:38.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.592+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:38.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.592+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:07:38.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.600+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:38.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:38.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:38.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:38.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:38.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:38.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.624+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:38.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.624+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:38.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:38.624+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:39.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:39.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:39.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:39.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:39.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:39.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:39.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:39.320+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:39.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:39.320+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:40.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.924+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:40.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:40.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:40.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:40.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:40.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.956+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:40.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.956+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:40.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:40.960+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:41.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.340+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:41.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:41.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:41.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:41.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:41.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:41.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.464+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:41.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.464+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:41.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.472+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:41.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.472+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:41.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:41.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:41.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:41.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:41.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:41.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.476+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:41.484 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:41.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:41.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:41.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:41.524 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:41.516+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:43.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.232+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:43.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:43.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:43.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:43.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.320+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:43.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.320+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:43.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.320+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:43.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:43.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:43.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:43.503 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:07:43.503 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:07:43.532 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:07:43.533 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:07:44.822 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2306948515 2023-10-14T00:07:44.825 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:07:44.826 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242064 2023-10-14T00:07:44.826 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:07:44.827 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2306948515 ]] 2023-10-14T00:07:44.827 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2306948515 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\5\5\7\4\3\7\3\6 ]] 2023-10-14T00:07:44.827 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2306948515 2023-10-14T00:07:45.139 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:45.136+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:45.146 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:45.144+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:45.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:45.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:45.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:45.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:45.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:45.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.552+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:45.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.552+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:45.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:45.552+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:46.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.132+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:46.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.132+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:46.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:46.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:46.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:46.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.192+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:46.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.192+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:46.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.196+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:46.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.224+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:46.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:46.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:46.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:46.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:46.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.284+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:46.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:46.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:46.564+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:47.759 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2306948515 until 2023-10-14T01:07:47.011405+0000 (3600 sec) 2023-10-14T00:07:47.831 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2306948515 2023-10-14T00:07:47.832 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:07:48.184 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:48.180+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:49.027 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:49.024+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:49.028 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:49.024+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:49.040 INFO:tasks.workunit.client.0.smithi183.stderr:listed 7 entries 2023-10-14T00:07:49.074 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2306948515 2023-10-14T00:07:49.074 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:07:49.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:49.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:49.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:49.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:49.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:49.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.352+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:49.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.352+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:49.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.380+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:49.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.384+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:07:49.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.384+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:07:49.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.384+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:49.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:49.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:49.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:49.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:49.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:49.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:49.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:49.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:49.420+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:50.247 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:50.244+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:07:50.256 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:50.252+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:50.315 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:50.312+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:50.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:50.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:50.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:50.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:50.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.528+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:50.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.528+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:50.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.556+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:50.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:50.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:50.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:50.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:50.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.844+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:50.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.844+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:50.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:50.848+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:54.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:54.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:54.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:54.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:54.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:54.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:54.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:54.084+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:54.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:54.084+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:54.717 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:54.716+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:54.723 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:54.720+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:55.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.420+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:55.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:55.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:55.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:55.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:55.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:55.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.464+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:55.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.464+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:55.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:55.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:55.507 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:55.500+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:55.508 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:55.500+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:56.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:56.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:56.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:56.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:56.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:56.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:56.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:56.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:56.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:56.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:56.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:56.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:56.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.352+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:56.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:56.352+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:57.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.148+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:57.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:57.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:57.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:57.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:57.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:57.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.184+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:57.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.184+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:57.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.184+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:57.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.372+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:57.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:57.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:57.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:57.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:57.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.420+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:57.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:57.420+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:58.973 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:58.972+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:07:58.979 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:07:58.976+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:07:59.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.056+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:07:59.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:07:59.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:07:59.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:07:59.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:07:59.085 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:07:59.085 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:07:59.096 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:07:59.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:07:59.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.096+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:07:59.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.100+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:07:59.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:07:59.100+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:07:59.106 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:08:00.019 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.016+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.020 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.016+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2ffa900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.028+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.028+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bac00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.028+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.028+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bad80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.036+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.036+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.036+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26baf80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.036+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bb180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.040+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.040+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bb380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.064+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.064+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bb980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.073 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.068+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.073 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.068+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bb800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bbf00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bbd80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bbb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.080+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb26bb600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.104 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.100+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.100+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.100+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.100+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbe180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.100+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbe380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.100+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbe580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.112+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.112+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbe780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.116+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.116+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbe980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.165 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.164+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.165 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.164+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbeb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.166 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.164+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.166 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.164+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.166 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.164+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbf580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.166 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:08:00.164+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbf400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:08:00.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:00.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:00.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:00.296+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:00.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:00.412+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:00.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:00.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:00.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:00.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:00.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:00.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:00.481 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2306948515 2023-10-14T00:08:00.481 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:08:00.482 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242080 2023-10-14T00:08:00.482 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:08:00.482 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2306948515 ]] 2023-10-14T00:08:00.483 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2306948515 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\3\0\6\9\4\8\5\1\5 ]] 2023-10-14T00:08:00.483 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:08:00.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:00.480+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:01.201 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:01.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:01.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:01.336+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:03.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.448+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:03.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:03.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:03.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:03.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:03.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:03.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.520+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:03.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.520+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:03.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:03.528+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:05.482 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:05.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:08:05.492 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:05.488+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:05.492 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:05.488+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:06.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:06.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:06.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:06.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:06.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.136+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:06.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.136+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:06.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.144+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:06.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.716+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:06.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:06.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:06.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:06.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:06.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.768+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:06.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.768+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:06.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:06.768+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:07.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.060+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:07.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:07.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:07.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:07.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:07.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:07.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.096+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:07.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.096+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:07.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.100+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:07.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:07.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:07.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:07.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:07.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.228+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:07.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.228+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:07.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:07.228+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:10.486 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:08:10.486 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:08:10.487 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:08:10.491 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:08:11.179 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:11.176+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:11.964 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2306948515 2023-10-14T00:08:11.964 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:08:11.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242091 2023-10-14T00:08:11.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:08:11.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2306948515 ]] 2023-10-14T00:08:11.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2306948515 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\3\0\6\9\4\8\5\1\5 ]] 2023-10-14T00:08:11.966 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:08:11.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:11.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:11.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:11.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:11.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:11.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:11.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:11.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:11.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:11.976+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:11.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:11.976+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:11.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:11.980+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:13.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:13.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:13.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:13.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:14.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:14.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:14.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:14.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:14.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.212+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:14.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.212+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:14.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.212+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:14.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:14.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:14.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:14.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:14.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.332+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:14.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.332+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:14.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:14.340+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:15.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:15.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:15.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:15.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:15.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:15.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.444+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:15.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.444+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:15.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:15.452+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:16.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.288+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:16.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:16.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:16.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:16.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:16.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.336+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:16.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.336+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:16.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:16.340+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:18.755 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:18.752+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:18.771 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:18.768+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:20.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.020+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:20.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:20.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:20.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:20.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:20.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.080+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:20.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.080+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:20.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.104+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:20.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.360+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:20.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:20.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:20.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:20.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:20.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:20.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.404+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:20.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.404+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:20.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:20.416+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:21.970 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:08:21.970 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:08:21.985 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:08:21.985 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:08:21.993 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:21.992+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:08:22.065 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:22.064+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:22.075 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:22.072+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:22.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:22.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:22.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:22.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:22.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:22.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:23.339 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2306948515 2023-10-14T00:08:23.343 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:08:23.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242103 2023-10-14T00:08:23.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:08:23.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2306948515 ]] 2023-10-14T00:08:23.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2306948515 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\3\0\6\9\4\8\5\1\5 ]] 2023-10-14T00:08:23.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:08:23.948 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:23.948+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:08:23.956 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:23.952+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:24.996 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:24.992+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:08:24.998 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:24.996+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:25.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:25.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:25.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:25.228+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:25.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:25.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:25.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:25.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:25.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:25.312+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:26.350 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:26.348+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:08:26.357 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:26.356+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:08:26.370 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:26.368+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:26.384 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:26.380+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:28.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:28.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:28.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:28.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:28.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.172+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:28.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.172+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:28.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.172+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:28.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:28.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:28.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:28.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:28.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:28.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:28.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.844+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:28.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.844+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:28.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:28.844+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:29.931 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:29.928+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:30.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.376+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:30.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:30.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:30.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:30.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:30.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:30.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.432+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:30.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.432+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:30.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:30.432+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:30.937 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:30.940+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:31.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:31.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:31.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:31.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:31.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:31.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:31.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:31.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:31.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:31.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:31.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:31.456+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:31.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:31.464+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:32.137 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:32.140+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:33.347 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:08:33.348 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:08:33.353 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:08:33.354 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:08:34.923 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2306948515 2023-10-14T00:08:34.923 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:08:34.929 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242114 2023-10-14T00:08:34.929 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:08:34.929 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2306948515 ]] 2023-10-14T00:08:34.930 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2306948515 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\3\0\6\9\4\8\5\1\5 ]] 2023-10-14T00:08:34.930 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:08:35.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.084+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:35.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:35.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:35.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:35.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:35.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:35.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.136+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:35.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.136+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:35.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.136+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:35.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:35.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:35.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:35.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.572+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:35.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.572+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:35.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:35.576+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:36.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.560+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:36.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:36.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:36.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:36.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:36.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:36.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.604+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:36.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.604+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:36.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.620+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:36.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.788+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:36.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:36.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:36.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:36.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:36.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:36.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.812+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:36.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.812+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:36.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.812+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:36.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.888+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:36.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:36.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:36.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:36.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:36.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.924+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:36.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.944+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:36.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:36.944+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:37.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.444+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:37.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:37.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:37.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:37.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:37.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.472+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:37.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.472+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:37.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:37.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:37.607 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:37.608+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:37.612 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:37.616+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:40.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:40.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:40.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:40.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:40.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:40.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:40.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.388+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:40.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.388+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:40.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:40.388+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:42.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.020+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:42.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:42.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:42.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:42.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:42.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:42.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.044+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:42.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.044+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:42.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.052+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:42.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.456+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:42.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:42.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:42.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:42.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:42.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:42.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:42.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:42.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.492+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:42.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:42.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:42.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:42.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:42.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:43.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:43.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:43.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:43.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:43.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:43.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:43.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:43.660+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:43.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:43.660+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:43.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:43.684+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:44.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.756+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:08:44.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.756+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:08:44.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.756+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:44.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:44.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:44.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:44.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:44.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:44.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.832+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:44.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.832+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:44.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:44.840+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:44.938 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:08:44.940 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:08:44.944 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:08:44.947 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:08:46.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/399520586 2023-10-14T00:08:46.124 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:08:46.126 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242126 2023-10-14T00:08:46.126 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:08:46.127 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/399520586 ]] 2023-10-14T00:08:46.127 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/399520586 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\3\0\6\9\4\8\5\1\5 ]] 2023-10-14T00:08:46.127 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/399520586 2023-10-14T00:08:46.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:46.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:46.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:46.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:46.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:46.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.592+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:46.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.592+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:46.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.592+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:46.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:46.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:46.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:46.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:46.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:46.792+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:48.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:48.248+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:48.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:48.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:48.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:48.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:48.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:48.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:48.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:48.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:48.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:48.264+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:48.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:48.264+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:48.299 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:48.300+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:48.325 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:48.328+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:48.797 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/399520586 until 2023-10-14T01:08:48.173014+0000 (3600 sec) 2023-10-14T00:08:48.846 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:08:48.846 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/399520586 2023-10-14T00:08:49.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.252+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:49.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:49.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:49.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:49.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:49.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:49.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.300+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:49.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.300+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:49.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:49.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:49.849 INFO:tasks.workunit.client.0.smithi183.stderr:listed 8 entries 2023-10-14T00:08:49.925 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/399520586 2023-10-14T00:08:49.926 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:08:50.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.012+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:50.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:50.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:50.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:50.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:50.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:50.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.036+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:50.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.036+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:50.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.036+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:50.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.300+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:50.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:50.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:50.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:50.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:50.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:50.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:50.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:50.378 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:50.380+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:08:50.390 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:50.392+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:08:50.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:50.392+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:51.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.036+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:08:51.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:51.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.084+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.084+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:51.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.084+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:51.689 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:08:51.688+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:08:51.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:51.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:51.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:51.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:51.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:51.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.748+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:51.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.748+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:51.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:51.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:55.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:55.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:55.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:55.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:55.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:55.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.864+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:55.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.864+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:55.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:55.876+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:56.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:56.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:56.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:56.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:56.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:56.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:56.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:56.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:56.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:56.344+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:56.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:56.344+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:56.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:56.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:58.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:58.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:58.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:58.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.647+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:58.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.647+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:58.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:58.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:58.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:58.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:08:58.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.883+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:08:58.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:58.883+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:08:59.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:59.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:08:59.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:59.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:08:59.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:59.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:08:59.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:59.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:08:59.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:08:59.287+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:08:59.928 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:08:59.928 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:08:59.930 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:08:59.931 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:09:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.023+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.023+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb295cb00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.023+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.023+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb1e1ad00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.027+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.027+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2931680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.027+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.027+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2930080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.027+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.027+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2931c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.054 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.055+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.054 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.055+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.055 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.055+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb26ba680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.056 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.055+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d29780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.072 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.075+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.073 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.075+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d29580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.079+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.083+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253c680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.107+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.107+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffa280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.107+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.107+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb275a700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.152 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.155+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.153 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.155+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2d29f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.160 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.163+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.160 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.163+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253c500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.161 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.163+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.162 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.163+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.162 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.163+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253c880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.162 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.163+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253ca80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.179 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.179+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.179 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.179+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253cc80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.181 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.183+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.181 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.183+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253ce80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.182 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.183+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.182 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.183+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.182 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.183+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253d080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.182 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.183+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253da80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.187 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.187+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.187 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:00.187+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253d900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:00.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.487+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:00.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:00.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:00.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:00.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:00.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.523+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:00.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.523+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:00.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.523+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:00.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:00.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:00.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:00.779+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:01.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.255+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:01.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:01.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:01.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:01.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:01.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:01.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.315+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:01.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.315+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:01.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.323+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:01.384 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/399520586 2023-10-14T00:09:01.391 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:09:01.393 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242141 2023-10-14T00:09:01.393 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:09:01.393 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/399520586 ]] 2023-10-14T00:09:01.393 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/399520586 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\9\9\5\2\0\5\8\6 ]] 2023-10-14T00:09:01.393 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:09:01.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.499+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:01.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:01.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:01.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:01.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.555+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:01.558 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:01.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:01.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.563+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:01.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.663+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:01.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:01.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:01.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:01.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:01.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:01.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.699+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:01.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.699+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:01.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:01.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:02.331 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:02.331+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:02.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:02.339+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:03.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:03.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:03.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:03.339+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:04.718 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:04.719+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:09:04.742 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:04.743+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:04.771 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:04.771+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:04.836 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:04.839+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:09:04.856 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:04.859+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:04.872 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:04.871+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:06.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:06.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:06.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:06.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:06.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:06.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.839+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:06.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.839+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:06.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:06.839+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:07.234 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:07.235+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:07.241 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:07.243+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:07.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:07.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:07.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:07.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:07.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:07.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.291+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:07.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.291+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:07.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:07.319+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:08.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:08.575+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:08.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:08.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:08.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:08.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:08.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:08.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:08.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:08.623+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:09.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.495+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:09.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:09.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:09.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:09.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:09.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:09.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.531+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:09.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.531+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:09.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:09.531+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:10.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:10.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:10.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:10.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:10.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:10.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:10.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:10.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:11.395 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:09:11.395 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:09:11.400 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:09:11.414 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:09:11.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.451+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:11.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:11.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:11.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:11.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:11.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:11.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.491+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:11.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.491+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:11.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:11.491+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:12.660 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/399520586 2023-10-14T00:09:12.664 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:09:12.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242152 2023-10-14T00:09:12.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:09:12.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/399520586 ]] 2023-10-14T00:09:12.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/399520586 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\9\9\5\2\0\5\8\6 ]] 2023-10-14T00:09:12.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:09:12.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:12.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:12.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:12.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:12.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:12.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:12.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:12.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:12.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:12.839+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:12.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:12.855+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:12.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:12.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:13.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:13.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:13.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:13.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:13.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:13.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.279+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:13.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.279+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:13.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.279+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:13.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.343+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:13.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.551+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:13.558 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:13.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:09:13.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:13.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:13.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:13.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:13.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:13.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.575+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:13.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.575+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:13.577 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:13.579+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:13.577 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:13.579+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:13.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:13.583+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:15.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:15.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:15.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:15.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:15.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:15.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.619+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:15.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.619+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:15.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.619+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:15.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.851+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:15.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:15.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:15.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:15.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:15.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:15.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.859+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:15.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.883+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:15.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:15.883+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:18.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:18.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:18.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:18.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.375+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:18.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:18.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:18.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:18.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:18.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.687+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:18.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:18.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:18.905 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:18.907+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:09:18.934 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:18.935+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:09:18.954 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:18.955+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:18.970 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:18.971+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:19.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.263+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:19.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:19.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:19.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:19.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:19.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.295+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:19.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.295+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:19.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:19.299+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:20.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.855+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:20.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:20.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:20.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:20.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:20.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:20.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.895+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:20.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.895+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:20.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:20.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:21.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:21.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:21.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.311+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:21.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:21.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:21.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.351+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:21.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.351+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:21.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.351+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:21.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.979+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:21.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:21.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:22.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:22.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:22.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:22.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.067+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:22.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.067+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:22.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.079+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:22.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:22.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:22.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.223+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:22.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.455+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:22.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:22.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:22.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:22.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:22.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.495+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:22.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.495+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:22.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:22.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:22.668 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:09:22.681 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:09:22.681 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:09:22.682 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:09:23.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:23.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:23.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:23.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:23.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:23.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:23.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:23.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:23.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:23.955+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:23.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:23.955+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:23.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:23.955+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:24.294 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/399520586 2023-10-14T00:09:24.296 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:09:24.297 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242164 2023-10-14T00:09:24.297 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:09:24.298 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/399520586 ]] 2023-10-14T00:09:24.298 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/399520586 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\9\9\5\2\0\5\8\6 ]] 2023-10-14T00:09:24.299 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:09:24.729 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:24.731+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:25.909 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:25.911+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:26.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:26.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:26.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:26.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:26.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:26.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.587+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:26.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.587+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:26.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:26.595+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:27.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:27.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:27.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:27.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:27.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:27.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:27.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:27.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:27.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:27.715+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:27.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:27.739+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:28.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:28.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:29.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:29.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:29.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:29.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:29.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.031+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:29.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.031+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:29.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.051+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:29.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:29.179+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:30.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.143+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:30.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:30.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:30.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:30.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:30.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:30.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.191+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:30.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.191+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:30.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:30.195+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:31.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:31.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:31.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:31.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:31.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:31.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.055+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:31.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.055+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:31.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:31.095+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:31.307 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:31.311+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:32.006 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:32.007+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:34.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:34.047 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:34.047+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:34.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:34.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:34.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.051+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:34.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.067+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:34.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:34.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:34.154 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:34.155+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:34.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:34.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:34.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:34.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.231+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:34.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.239+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:34.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.247+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:34.307 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:09:34.307 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:09:34.311 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:09:34.324 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:09:34.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:34.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:34.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:34.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:34.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:34.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.835+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:34.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.835+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:34.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:34.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:35.594 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/399520586 2023-10-14T00:09:35.595 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:09:35.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242175 2023-10-14T00:09:35.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:09:35.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/399520586 ]] 2023-10-14T00:09:35.596 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/399520586 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\9\9\5\2\0\5\8\6 ]] 2023-10-14T00:09:35.596 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:09:36.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.343+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:36.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:36.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:36.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:36.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:36.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:36.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.407+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:36.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.407+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:36.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:36.415+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:37.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:37.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:37.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:37.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:37.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.883+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:37.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.883+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:37.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.887+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:37.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:37.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:38.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:38.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:38.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:38.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:38.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.031+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:38.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.031+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:38.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.031+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:38.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:38.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:38.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:38.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:38.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.627+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:38.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.627+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:38.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:38.639+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:39.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.155+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:39.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:39.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:39.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:39.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:39.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.175+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:39.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.175+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:39.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.175+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:39.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:39.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:39.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:39.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:39.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:39.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.843+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:39.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.843+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:39.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:39.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:40.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:40.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:40.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:40.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:40.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:40.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:40.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:40.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:40.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:40.303+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:40.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:40.303+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:40.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:40.303+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:42.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.311+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:42.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:42.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:42.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:42.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.343+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:42.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:42.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:42.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:42.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.691+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:42.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.691+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:42.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:42.699+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:43.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:43.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:43.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:43.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:43.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:43.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:43.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:43.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:43.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:43.627+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:43.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:43.627+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:43.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:43.631+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:45.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:45.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:45.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:45.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:45.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.223+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:45.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:45.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:45.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:45.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:45.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:45.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.371+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:45.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.371+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:45.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:45.387+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:45.619 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:09:45.620 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:09:45.625 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:09:45.627 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:09:45.983 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:45.987+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:46.015 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:46.015+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:46.928 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/399520586 2023-10-14T00:09:46.940 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:09:46.948 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242186 2023-10-14T00:09:46.948 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:09:46.948 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/399520586 ]] 2023-10-14T00:09:46.948 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/399520586 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\9\9\5\2\0\5\8\6 ]] 2023-10-14T00:09:46.949 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:09:47.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:47.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:47.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:47.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:47.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:47.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:47.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:47.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:47.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:47.859+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:47.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:47.859+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:47.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:47.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:49.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:49.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:49.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:49.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:49.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:49.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.135+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:49.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.135+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:49.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.135+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:49.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:49.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:49.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:49.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:49.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.491+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:49.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.491+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:49.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:49.491+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:50.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.259+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:50.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:50.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:50.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:50.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.303+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:50.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.303+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:50.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.307+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:50.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:50.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:50.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:50.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:50.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.635+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:50.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.635+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:50.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:50.675+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:51.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:51.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:51.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:51.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:51.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:51.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:51.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:51.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:51.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:51.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:51.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:51.479+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:52.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.447+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:09:52.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:52.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:52.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:52.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:52.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:52.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.479+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:52.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.479+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:52.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:52.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:54.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:54.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:54.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:54.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:54.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:54.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.931+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:54.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.931+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:09:54.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:54.931+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:55.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:55.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:09:55.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:55.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:09:55.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:55.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:09:55.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:55.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:09:55.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:55.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:09:55.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:55.387+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:09:55.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:55.387+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:56.590 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:56.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:09:56.613 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:56.615+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:09:56.614 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:56.615+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:56.619 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:56.623+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:56.775 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:56.775+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:56.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:09:56.875+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:09:56.975 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:09:56.978 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:09:56.979 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:09:56.992 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:09:57.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:57.403+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:09:57.422 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:09:57.423+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:09:58.488 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/301131838 2023-10-14T00:09:58.488 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:09:58.489 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242198 2023-10-14T00:09:58.489 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:09:58.489 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/301131838 ]] 2023-10-14T00:09:58.490 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/301131838 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\9\9\5\2\0\5\8\6 ]] 2023-10-14T00:09:58.490 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/301131838 2023-10-14T00:09:59.909 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:59.911+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb1f99c80 handle_get_snapshots: failed to get snapshots: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:59.926 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:59.927+0000 7f660e1ef640 -1 librbd::mirror::GetInfoRequest: 0x55ccb1f99bc0 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:59.941 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:59.943+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb1f98540 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:59.979 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:59.983+0000 7f660e1ef640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5f500 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:59.989 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:59.991+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5f740 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:09:59.993 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:09:59.995+0000 7f660e1ef640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5f800 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.003+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5f980 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.011+0000 7f660e1ef640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5f380 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.011 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.015+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5e480 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.011 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.015+0000 7f660e1ef640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5fe00 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.012 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.015+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5f8c0 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.012 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.015+0000 7f660e1ef640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2d5f140 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.012 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.015+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb29252c0 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.019+0000 7f660e1ef640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2925980 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.016 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:10:00.019+0000 7f660a9e8640 -1 librbd::api::Mirror: image_info_list: failed to list mirror image statuses: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:00.865 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/301131838 until 2023-10-14T01:10:00.130218+0000 (3600 sec) 2023-10-14T00:10:00.920 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:10:00.927 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/301131838 2023-10-14T00:10:01.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.663+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:01.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:01.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:01.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:01.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:01.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:01.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.775+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:01.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.775+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:01.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:01.775+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:02.232 INFO:tasks.workunit.client.0.smithi183.stderr:listed 9 entries 2023-10-14T00:10:02.293 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/301131838 2023-10-14T00:10:02.293 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:10:02.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:02.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:02.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:02.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:02.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:02.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:02.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:02.879+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:02.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:02.931+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:02.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:02.931+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:03.505 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:03.507+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:04.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:04.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:04.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:04.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:04.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.259+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:04.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.259+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:04.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.267+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:04.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.435+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:04.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:04.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:04.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:04.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:04.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:04.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.499+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:04.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.499+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:04.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:04.511+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:05.145 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:05.147+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:05.151 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:05.151+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:05.315 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:05.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:10:05.330 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:05.331+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:05.343 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:05.343+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:05.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:05.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:05.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:05.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:05.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:05.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:05.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:05.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:05.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:05.551+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:05.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:05.559+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:07.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:07.939+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:08.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:08.126 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:08.127+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:08.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:08.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:08.152 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:08.151+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:08.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:08.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:08.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.183+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:08.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.183+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:08.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:08.191+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:10.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.459+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:10.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:10.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:10.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:10.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:10.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.495+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:10.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.495+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:10.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.507+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:10.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.867+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:10.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:10.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:10.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:10.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:10.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:10.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:10.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:11.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:11.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:11.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:11.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.159+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:11.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.531+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:11.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:11.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:11.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:11.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:11.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:11.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.559+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:11.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.559+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:11.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:11.559+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:11.652 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:11.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:11.654 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:11.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:12.301 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:10:12.302 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:10:12.304 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:10:12.304 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:10:12.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.491+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:12.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:12.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:12.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:12.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:12.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.583+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:12.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.583+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:12.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:12.587+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:13.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1692311500 2023-10-14T00:10:13.713 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:10:13.713 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242213 2023-10-14T00:10:13.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:10:13.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1692311500 ]] 2023-10-14T00:10:13.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1692311500 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\0\1\1\3\1\8\3\8 ]] 2023-10-14T00:10:13.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/1692311500 2023-10-14T00:10:15.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:15.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:15.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:15.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:15.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:15.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:15.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:15.503+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:15.814 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/1692311500 until 2023-10-14T01:10:15.149960+0000 (3600 sec) 2023-10-14T00:10:15.868 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:10:15.871 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/1692311500 2023-10-14T00:10:16.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.403+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:16.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:16.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:16.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:16.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:16.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:16.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.455+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:16.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.455+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:16.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:16.455+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:16.992 INFO:tasks.workunit.client.0.smithi183.stderr:listed 10 entries 2023-10-14T00:10:17.048 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/1692311500 2023-10-14T00:10:17.048 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:10:17.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:17.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:17.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:17.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.299+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:17.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.299+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:17.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.319+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:17.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:17.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:17.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:17.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:17.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:17.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:17.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:17.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:17.911+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:18.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.111+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:10:18.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.159+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:18.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:18.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:18.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:18.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:18.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:18.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.167+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:18.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.167+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:18.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:18.171+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:19.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:19.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:19.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:19.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:19.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:19.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.495+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:19.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.495+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:19.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:19.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:21.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:21.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:21.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:21.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:21.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:21.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:21.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:21.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:21.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:21.211+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:21.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:21.211+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:21.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:21.211+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:22.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:22.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:22.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:22.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:22.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:22.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.375+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:22.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.375+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:22.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:22.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.939+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:22.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:22.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:22.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:22.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:22.987+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:24.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.579+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:24.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:24.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:24.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:24.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:24.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.595+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:24.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.595+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:24.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:24.595+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:25.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:25.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:25.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:25.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:25.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:25.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:25.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:25.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:25.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:25.087+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:25.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:25.103+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:25.164 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:25.167+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:25.181 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:25.183+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:25.362 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:25.363+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:26.245 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:26.247+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:27.050 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:10:27.051 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:10:27.060 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:10:27.077 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:10:27.234 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:27.239+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:10:27.274 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:27.279+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:28.140 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1692311500 2023-10-14T00:10:28.142 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:10:28.143 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242228 2023-10-14T00:10:28.143 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:10:28.144 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1692311500 ]] 2023-10-14T00:10:28.144 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1692311500 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\6\9\2\3\1\1\5\0\0 ]] 2023-10-14T00:10:28.144 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:10:29.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:29.715+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:29.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:29.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:29.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:29.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:29.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:29.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:29.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:29.723+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:30.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:30.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:30.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:30.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:30.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:30.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.283+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:30.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.283+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:30.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.319+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:30.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.319+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:10:30.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.319+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:10:30.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:30.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:30.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:30.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:30.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.351+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:30.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.351+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:30.615 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:30.619+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:30.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:30.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:30.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:30.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:30.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:30.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.851+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:30.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:30.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:31.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:31.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:31.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:31.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:31.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:31.515+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:34.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:34.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:34.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:34.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:34.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.415+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:34.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.475+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:34.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:34.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:34.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:34.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:34.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:34.515+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:34.701 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:34.703+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:10:34.712 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:34.715+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:35.717 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:35.719+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:10:35.723 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:35.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:35.723 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:35.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:35.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:35.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:35.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:35.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:35.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:35.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:35.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:35.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:35.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:36.038 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:36.039+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:10:36.042 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:36.039+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:37.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.627+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:37.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:37.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:37.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:37.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:37.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:37.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.639+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:37.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.639+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:37.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:37.639+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:38.058 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:38.063+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:38.145 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:10:38.145 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:10:38.146 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:10:38.148 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:10:38.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.495+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:38.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:38.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:38.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:38.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:38.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.563+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:38.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.583+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:38.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.595+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:38.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:38.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:38.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:38.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:38.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:39.670 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1692311500 2023-10-14T00:10:39.670 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:10:39.670 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242239 2023-10-14T00:10:39.671 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:10:39.671 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1692311500 ]] 2023-10-14T00:10:39.671 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1692311500 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\6\9\2\3\1\1\5\0\0 ]] 2023-10-14T00:10:39.671 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:10:41.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.367+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:41.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:41.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:41.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:41.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:41.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:41.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.387+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:41.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.387+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:41.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:41.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.391+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:10:41.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.391+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:41.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:41.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:41.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:41.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:41.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:41.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.423+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:41.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.775+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:41.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:41.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:41.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:41.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:41.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:41.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:41.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:41.851+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:42.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:42.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:42.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:42.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:42.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:42.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:42.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:42.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:42.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:42.751+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:45.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:45.303+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:45.329 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:45.331+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:45.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:45.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:45.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:45.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:45.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.387+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:45.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.411+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:45.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.635+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:45.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:45.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:45.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:45.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:45.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.687+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:45.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.687+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:45.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.687+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:45.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:45.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:45.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:45.959+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:46.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:46.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:46.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:46.395+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:46.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:46.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:47.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.363+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:10:47.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.363+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:10:47.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.367+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:47.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:47.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:47.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:47.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:47.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:47.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.439+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:47.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.439+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:47.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.439+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:47.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:47.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:47.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:47.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:47.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:47.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:47.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:47.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:47.923+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:48.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:48.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:48.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:48.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:48.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:48.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:48.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:48.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:49.672 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:10:49.676 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:10:49.677 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:10:49.690 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:10:51.335 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1692311500 2023-10-14T00:10:51.336 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:10:51.338 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242251 2023-10-14T00:10:51.338 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:10:51.338 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1692311500 ]] 2023-10-14T00:10:51.339 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1692311500 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\6\9\2\3\1\1\5\0\0 ]] 2023-10-14T00:10:51.339 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:10:51.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:51.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:51.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:51.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:51.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:51.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:51.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:51.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:51.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:51.866+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:51.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:51.866+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:52.390 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:52.390+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:52.390 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:52.390+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:52.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.438+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:10:52.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.438+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:10:52.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.446+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:52.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:52.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:52.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:52.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:52.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:52.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.486+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:52.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.486+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:52.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:52.486+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:52.952 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:52.950+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:54.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.094+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:54.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:54.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:54.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:54.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:54.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.130+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:54.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.142+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:54.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.158+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:54.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.202+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:10:54.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.206+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:54.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:54.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:54.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:54.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:54.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:54.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:54.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:54.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:54.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:54.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:54.576 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:54.578+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:10:54.576 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:10:54.578+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:10:56.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.274+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:56.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:56.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:56.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:56.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:56.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:56.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.330+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:56.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.330+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:56.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.346+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:56.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.462+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:56.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:56.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:56.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:56.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:56.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:56.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.490+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:56.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.490+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:56.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.506+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:56.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.586+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:56.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:56.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:56.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:56.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:56.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:56.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.694+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:56.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.694+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:56.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:56.698+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:57.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:57.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:10:57.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:57.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:57.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:57.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:57.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:57.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:57.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:57.090+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:57.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:57.090+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:57.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:57.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:59.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.470+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:10:59.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:10:59.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:10:59.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:10:59.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:10:59.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.486+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:10:59.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.486+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:10:59.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:10:59.486+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:10:59.999 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.002+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:10:59.999 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.002+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb275a780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.003 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.006+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.004 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.006+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb25d5800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.018+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.018+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb26ba700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.017 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.018+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.018 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.018+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c3ba80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.042+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.042+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb26bb880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.051 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.054+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.051 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.054+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253d780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.054+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.054+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2fbe480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.066+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.066+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253c980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.086+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.085 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.086+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2931b80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.085 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.086+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.085 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.086+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d29e80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.086 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.090+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.086 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.090+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.090+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2fbe780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.090+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2fbe680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.102+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.102+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d92f80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.106+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.106+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d92d80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.106+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.103 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.106+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2931b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.114+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.114+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.134+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.134+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.139 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.142+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.139 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.142+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.149 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.150+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.149 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.150+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.164 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.166+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.164 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:11:00.166+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93a80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:11:00.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.714+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:00.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.722+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:11:00.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.734+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:00.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:00.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:00.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:00.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:00.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:00.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.770+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:00.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.770+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:00.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:00.794+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:01.001 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:11:01.002+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:11:01.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:01.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:01.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:01.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:01.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:01.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:01.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:01.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:01.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:01.126+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:01.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:01.126+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:01.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:01.126+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:01.357 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:11:01.358 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:11:01.358 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:11:01.362 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:11:01.482 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:01.486+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:02.850 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1692311500 2023-10-14T00:11:02.860 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:11:02.861 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242262 2023-10-14T00:11:02.861 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:11:02.861 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1692311500 ]] 2023-10-14T00:11:02.861 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1692311500 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\6\9\2\3\1\1\5\0\0 ]] 2023-10-14T00:11:02.862 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:11:03.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:03.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:05.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:05.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:05.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:05.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:05.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:05.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.362+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:05.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.362+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:05.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.366+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:05.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.862+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:05.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:05.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:05.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:05.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:05.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:05.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.882+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:05.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.882+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:05.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:05.882+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:06.440 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:06.442+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:11:06.446 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:06.450+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:09.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:09.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:09.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:09.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:09.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:09.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:09.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:09.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.858+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:09.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.858+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:09.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:10.754 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:10.758+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:11:10.787 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:10.786+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:12.863 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:11:12.863 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:11:12.866 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:11:12.873 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:11:13.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.098+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:13.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:13.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:13.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:13.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:13.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.162+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:13.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.178+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:13.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.178+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:13.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.294+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:13.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:13.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:13.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:13.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:13.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.374+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:13.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.390+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:13.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:13.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:13.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:13.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:13.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:13.406+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:14.147 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1692311500 2023-10-14T00:11:14.149 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:11:14.150 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242274 2023-10-14T00:11:14.151 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:11:14.151 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1692311500 ]] 2023-10-14T00:11:14.151 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1692311500 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\6\9\2\3\1\1\5\0\0 ]] 2023-10-14T00:11:14.151 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:11:17.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:17.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:17.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:17.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:17.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:17.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:17.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:17.950+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:17.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:17.950+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:17.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:17.974+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:18.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.110+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:18.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.110+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:11:18.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.110+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:18.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:18.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:18.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:18.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:18.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:18.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.142+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:18.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.142+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:18.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:18.150+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:19.139 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:19.142+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:11:19.154 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:19.154+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:22.337 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:22.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:22.443 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:22.446+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:23.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:23.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:23.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:23.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:23.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.814+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:23.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.818+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:23.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.834+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:23.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.834+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:11:23.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.846+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:23.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:23.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:23.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:23.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:23.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:23.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.886+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:23.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:23.886+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:23.926 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:23.930+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:24.161 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:11:24.168 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:11:24.178 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:11:24.187 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:11:24.613 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:24.614+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:11:24.650 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:24.654+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:24.663 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:24.662+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:25.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:25.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:25.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:25.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:25.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:25.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.502+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:25.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.502+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:25.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:25.522+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:25.547 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3799504759 2023-10-14T00:11:25.547 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:11:25.548 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242285 2023-10-14T00:11:25.548 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:11:25.548 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3799504759 ]] 2023-10-14T00:11:25.548 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3799504759 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\6\9\2\3\1\1\5\0\0 ]] 2023-10-14T00:11:25.549 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/3799504759 2023-10-14T00:11:26.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:26.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:26.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:26.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:26.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:26.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:26.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:26.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:26.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:26.538+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:26.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:26.538+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:26.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:26.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:27.823 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:27.826+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:27.824 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/3799504759 until 2023-10-14T01:11:27.236785+0000 (3600 sec) 2023-10-14T00:11:27.881 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/3799504759 2023-10-14T00:11:27.893 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:11:29.086 INFO:tasks.workunit.client.0.smithi183.stderr:listed 11 entries 2023-10-14T00:11:29.128 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/3799504759 2023-10-14T00:11:29.129 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:11:29.611 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:29.614+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:11:29.623 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:29.626+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:30.215 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:30.218+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:11:30.233 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:30.234+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:30.237 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:30.242+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:30.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:30.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:30.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:30.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:31.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:31.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:31.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:31.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:31.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:31.018+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:31.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:31.062+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:31.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:31.062+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:31.969 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:31.970+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:35.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:35.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:35.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:35.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:35.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:35.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:35.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:35.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:35.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.822+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:35.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:35.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:35.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:35.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:35.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:35.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.886+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:35.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.886+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:35.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:35.886+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:36.908 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:36.910+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:37.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:37.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:37.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:37.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:37.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:37.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.638+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:37.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.638+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:37.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.694+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:37.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:37.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:37.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:37.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:37.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.838+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:37.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.838+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:37.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:37.842+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:38.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:38.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:38.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:38.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:38.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:38.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:38.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.082+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:38.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:38.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:38.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:38.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:38.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.234+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:38.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.234+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:38.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.234+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:38.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:38.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:38.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:38.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:38.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.942+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:38.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.982+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:38.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:38.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:39.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:39.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:39.131 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:11:39.132 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:11:39.133 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:11:39.142 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:11:39.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.594+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:39.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:39.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:39.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:39.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:39.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.614+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:39.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.614+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:39.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:39.614+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:40.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:40.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:40.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:40.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:40.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:40.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:40.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:40.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:40.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:40.090+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:40.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:40.090+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:40.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:40.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:40.447 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3799504759 2023-10-14T00:11:40.450 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:11:40.451 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242300 2023-10-14T00:11:40.451 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:11:40.451 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3799504759 ]] 2023-10-14T00:11:40.452 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3799504759 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\9\9\5\0\4\7\5\9 ]] 2023-10-14T00:11:40.452 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:11:41.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.802+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:41.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:41.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:41.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:41.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:41.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:41.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.846+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:41.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.854+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:41.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:41.882+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:42.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.478+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:42.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:42.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:42.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:42.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:42.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:42.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.490+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:42.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.490+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:42.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:42.498+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:44.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:44.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:44.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:44.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:44.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:44.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.106+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:44.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.106+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:44.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.106+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:44.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:44.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:44.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:44.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:44.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:44.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.506+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:44.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.506+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:44.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:44.506+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:45.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:45.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:45.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:45.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:45.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.178+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:45.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.222+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:45.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.222+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:11:45.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.226+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:45.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:45.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:45.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:45.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:45.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:45.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:45.306+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:49.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.210+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:49.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.210+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:11:49.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.214+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:49.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:49.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:49.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:49.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:49.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:49.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.230+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:49.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.230+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:49.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:49.238+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:50.467 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:11:50.468 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:11:50.473 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:11:50.503 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:11:51.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:51.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:51.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:51.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:51.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:51.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:51.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:51.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:51.766 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3799504759 2023-10-14T00:11:51.774 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:11:51.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242311 2023-10-14T00:11:51.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:11:51.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3799504759 ]] 2023-10-14T00:11:51.777 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3799504759 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\9\9\5\0\4\7\5\9 ]] 2023-10-14T00:11:51.777 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:11:52.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:52.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:52.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:52.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:52.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:52.342+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:52.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:52.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:52.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:52.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:53.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:53.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:53.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.018+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:53.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.018+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:53.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.022+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:53.049 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:53.054+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:53.178 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:53.182+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:11:53.193 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:53.194+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:53.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:53.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:53.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:53.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:53.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:53.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.654+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:53.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:53.658+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:53.938 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:53.938+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:11:53.938 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:53.938+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:53.938 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:53.938+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:54.589 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:54.594+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:55.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.238+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:55.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:55.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:55.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:55.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:55.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:55.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.286+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:55.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.286+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:55.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.290+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:55.316 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:55.314+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:55.324 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:55.326+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:55.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.498+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:55.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.518+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:55.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:55.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:55.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:55.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:55.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:55.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.614+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:55.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.614+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:55.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.622+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:55.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.658+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:55.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.658+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:11:55.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.674+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:55.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:55.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:55.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:55.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:55.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:55.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.718+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:55.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.718+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:55.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:55.718+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:56.785 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:56.786+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:11:56.822 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:56.826+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:56.836 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:56.838+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:56.920 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:56.922+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:11:57.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:57.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:57.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:57.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.130+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:57.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.130+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:57.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.146+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:57.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.622+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:11:57.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.658+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:57.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:57.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:57.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:57.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:57.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:11:57.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.714+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:11:57.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.714+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:11:57.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.714+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:57.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:57.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:57.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:11:57.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:57.770+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:11:57.794 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:57.798+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:57.865 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:57.870+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:58.105 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:58.106+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:11:58.117 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:11:58.122+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:11:59.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:59.234+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:11:59.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:59.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:11:59.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:59.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:11:59.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:59.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:11:59.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:11:59.242+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:00.005 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.006+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.005 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.006+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3179b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.006 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.010+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.006 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.010+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3179d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.014+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.014+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3179f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.016 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.018+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.016 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.018+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ee100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.030+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.030+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ee300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.029 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.030+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.029 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.030+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31eeb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.042+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.042+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31eea00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.053 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.058+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.054 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.058+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ee800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.054 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.058+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.055 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.058+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ee600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.055 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.058+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.055 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.058+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31eed00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.066+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.066+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31eef00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.082+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.082+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ef100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.086+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.086+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.086+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.086+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ef300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.086+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ef500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.086+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ef700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.098+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.098+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31ef900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.098+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.098+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31efb00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.118+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.118+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31efd00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.118+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.118+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.118+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb31eff00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:12:00.118+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3212100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:12:00.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:00.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:00.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:00.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:00.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:00.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:00.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:00.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:00.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:00.782+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:01.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:01.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:01.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:01.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:01.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:01.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:01.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:01.730+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:01.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:01.730+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:01.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:01.754+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:01.778 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:12:01.786 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:12:01.791 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:12:01.797 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:12:03.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.066+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:03.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:03.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:03.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.106+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:03.215 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:03.218+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:12:03.226 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:03.226+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:03.264 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3799504759 2023-10-14T00:12:03.264 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:12:03.266 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242323 2023-10-14T00:12:03.266 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:12:03.266 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3799504759 ]] 2023-10-14T00:12:03.267 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3799504759 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\9\9\5\0\4\7\5\9 ]] 2023-10-14T00:12:03.267 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:12:03.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:03.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:03.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:03.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:03.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.666+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:03.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.666+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:03.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:03.682+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:04.063 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:04.066+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:05.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:05.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:05.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:05.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:05.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:05.217+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:07.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:07.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:07.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:07.393+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:07.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:07.725+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:09.234 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:09.237+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:10.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.421+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:10.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:10.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:10.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:10.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:10.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:10.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.449+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:10.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.449+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:10.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:10.453+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:11.831 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:11.833+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:11.973 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:11.973+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:12.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:12.301+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:13.193 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:13.189+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:13.283 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:12:13.285 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:12:13.289 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:12:13.301 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:12:14.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:14.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:14.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:14.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:14.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:14.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:14.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:14.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:14.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:14.609+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:14.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:14.609+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:14.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:14.609+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:14.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3799504759 2023-10-14T00:12:14.779 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:12:14.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242334 2023-10-14T00:12:14.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:12:14.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3799504759 ]] 2023-10-14T00:12:14.781 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3799504759 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\9\9\5\0\4\7\5\9 ]] 2023-10-14T00:12:14.781 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:12:15.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:15.693+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:15.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:15.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:15.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:15.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:15.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:15.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:15.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:15.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:15.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:15.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:15.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:15.725+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:16.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:16.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:16.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:16.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:16.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:16.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:16.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:16.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:16.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:16.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:16.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:16.385+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:18.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.473+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:12:18.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:18.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:18.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:18.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:18.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:18.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.545+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:18.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.545+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:18.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:18.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:19.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:19.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:19.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:19.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:19.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:19.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:19.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:19.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:19.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:19.837+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:19.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:19.837+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:19.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:19.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:20.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.005+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:20.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:20.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:20.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.037+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:20.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:20.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:20.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:20.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:20.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:20.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:20.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.573+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:20.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:20.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:20.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:20.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:20.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.705+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:20.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:20.705+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:20.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:20.873+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:12:20.878 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:20.881+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:12:20.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:20.889+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:21.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.337+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:21.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:21.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:21.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:21.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:21.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.361+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:21.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.361+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:21.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.361+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:21.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.861+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:21.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:21.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:21.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:21.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.881+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:21.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.881+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:21.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:21.881+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:22.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:22.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:22.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:22.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:22.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:22.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.245+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:22.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.245+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:22.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.249+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:22.411 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:22.413+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:22.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:22.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:22.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:22.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.517+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:22.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:22.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:22.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:22.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:22.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.781+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:22.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.781+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:22.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:22.793+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:23.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:23.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:23.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:23.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:23.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.125+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:23.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.125+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:23.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.129+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:23.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:23.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:23.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:23.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:23.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.325+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:23.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.325+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:23.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:23.325+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:24.280 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:24.281+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:12:24.311 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:24.313+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:24.787 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:12:24.787 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:12:24.788 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:12:24.812 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:12:25.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.825+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:25.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:25.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:25.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:25.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:25.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.877+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:25.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.877+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:25.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:25.885+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:26.227 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3799504759 2023-10-14T00:12:26.241 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:12:26.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242346 2023-10-14T00:12:26.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:12:26.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3799504759 ]] 2023-10-14T00:12:26.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3799504759 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\9\9\5\0\4\7\5\9 ]] 2023-10-14T00:12:26.243 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:12:27.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:27.213+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:27.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:27.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:27.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:27.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:27.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:27.241+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:29.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:29.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:29.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:29.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:29.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:29.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.937+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:29.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.937+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:29.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:29.945+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:30.287 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:30.289+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:12:30.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:30.305+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:30.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:30.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:30.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:30.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:30.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:30.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:30.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:30.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:30.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:30.501+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:30.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:30.501+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:30.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:30.537+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:31.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:31.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:31.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:31.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:31.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:31.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:31.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.505+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:31.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:31.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:31.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:31.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:31.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.885+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:31.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.885+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:31.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:31.893+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:33.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:33.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:33.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:33.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:33.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:33.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.045+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:33.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.045+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:33.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.081+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:33.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:33.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:33.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:33.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:33.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:33.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.821+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:33.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.821+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:33.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:33.849+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:34.395 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:34.397+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:12:35.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.513+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:35.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:35.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:35.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:35.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:35.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:35.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.541+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:35.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.541+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:35.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:35.557+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:36.260 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:12:36.261 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:12:36.270 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:12:36.271 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:12:36.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:36.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:36.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:36.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:36.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:36.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.537+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:36.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.537+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:36.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:36.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:36.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:36.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:36.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:36.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.773+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:36.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.773+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:36.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:36.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.917+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:36.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:36.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:36.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:36.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:36.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:36.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.969+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:36.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.969+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:36.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:36.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:37.711 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4182509406 2023-10-14T00:12:37.711 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:12:37.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242357 2023-10-14T00:12:37.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:12:37.713 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4182509406 ]] 2023-10-14T00:12:37.713 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4182509406 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\7\9\9\5\0\4\7\5\9 ]] 2023-10-14T00:12:37.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/4182509406 2023-10-14T00:12:38.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.472+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:38.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:38.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:38.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:38.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:38.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:38.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.496+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:38.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.496+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:38.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.496+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:38.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.756+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:38.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:38.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:38.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:38.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:38.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:38.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.824+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:38.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.824+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:38.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:38.840+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:39.863 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/4182509406 until 2023-10-14T01:12:39.227098+0000 (3600 sec) 2023-10-14T00:12:39.904 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:12:39.930 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/4182509406 2023-10-14T00:12:40.531 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:40.532+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:40.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:40.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:40.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:40.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:40.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:40.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:40.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:40.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:40.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:40.952+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:40.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:40.952+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:40.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:40.968+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:41.057 INFO:tasks.workunit.client.0.smithi183.stderr:listed 12 entries 2023-10-14T00:12:41.131 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/4182509406 2023-10-14T00:12:41.131 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:12:44.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:44.332+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:44.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:44.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:44.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:44.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:44.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:44.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:44.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:44.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:44.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:44.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:44.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:44.352+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:48.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:48.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:48.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:48.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:49.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.048+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:49.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:49.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:49.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:49.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:49.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:49.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.068+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:49.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.068+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:49.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.068+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:49.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:49.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:49.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:49.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.816+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:49.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:49.816+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:51.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:51.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:51.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:51.145 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:12:51.145 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:12:51.146 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:12:51.147 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:12:51.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:51.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:51.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.184+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:51.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.184+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:51.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:51.192+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:52.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.164+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:52.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:52.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:52.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:52.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:52.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:52.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.236+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:52.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.236+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:52.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.248+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:52.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.528+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:52.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:52.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:52.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:52.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:52.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:52.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.572+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:52.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:52.572+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:52.717 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4182509406 2023-10-14T00:12:52.717 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:12:52.718 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242372 2023-10-14T00:12:52.718 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:12:52.718 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4182509406 ]] 2023-10-14T00:12:52.719 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4182509406 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\2\5\0\9\4\0\6 ]] 2023-10-14T00:12:52.719 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:12:53.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:53.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:53.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:53.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.492+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:53.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:53.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:53.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:53.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:53.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:54.025 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:54.024+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:12:54.062 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:12:54.064+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:12:54.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:54.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:54.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:54.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:54.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:54.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:54.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:54.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:54.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:54.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:55.607 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:12:55.608+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:12:57.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.848+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:57.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:57.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:12:57.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:57.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:57.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:57.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.932+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:57.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.932+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:57.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:57.932+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:12:58.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.212+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:12:58.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:12:58.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:12:58.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:12:58.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:12:58.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.252+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:12:58.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.252+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:12:58.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:12:58.252+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:00.003 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.004+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.004 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.004+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3212300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.012+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.011 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.012+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3212d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.017 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.016+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.017 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.016+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3212f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.024+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.024+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.024+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.024+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.024+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.024+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.039 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.040+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.039 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.040+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.044+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.044+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.057 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.056+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.057 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.056+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.064+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.064+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.064+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.064+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3213f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.072 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.072+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.072 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.072+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb325a100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.076+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.076+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb325a300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.080+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.080+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb325a500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.084+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.084+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325a700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.096+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.096+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325a900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.100+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.100+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325ab00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.108+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.108+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325ad00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.116+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.116+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.120+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325af00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:13:00.120+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb325b100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:13:00.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.268+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:00.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:00.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:00.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:00.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:00.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:00.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:00.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.340+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:00.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.340+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:00.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.384+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:00.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:00.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:00.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:00.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:00.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:00.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.444+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:00.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.444+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:00.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:00.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.824+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:00.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:00.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:00.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:00.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:00.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.852+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:00.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.852+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:00.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:00.880+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:02.722 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:13:02.723 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:13:02.726 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:13:02.756 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:13:03.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:03.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:03.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:03.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:03.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:03.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:03.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:03.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:03.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:03.836+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:03.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:03.836+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:03.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:03.836+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:04.226 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4182509406 2023-10-14T00:13:04.230 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:13:04.234 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242384 2023-10-14T00:13:04.234 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:13:04.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4182509406 ]] 2023-10-14T00:13:04.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4182509406 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\2\5\0\9\4\0\6 ]] 2023-10-14T00:13:04.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:13:04.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:04.464+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:04.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:04.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:04.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:04.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:04.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:04.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:04.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:04.492+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:05.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:05.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:06.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:06.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:06.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:06.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.024+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:06.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.032+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:06.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.032+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:06.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:06.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:06.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:06.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:06.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.048+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:06.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.072+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:06.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:06.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:06.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:06.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:06.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:06.704+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:07.283 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:07.288+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:07.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.748+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:07.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:07.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:07.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:07.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:07.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.772+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:07.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.772+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:07.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:07.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:09.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:09.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:09.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:09.616+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:09.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:09.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:09.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:09.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:09.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:09.628+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:09.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:09.628+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:10.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.052+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:10.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:10.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:10.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:10.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:10.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:10.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.080+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:10.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:10.080+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:12.521 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:12.524+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:12.537 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:12.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:12.589 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:12.596+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:12.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:12.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:12.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:12.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:12.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:12.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:12.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:12.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:12.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:12.816+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:12.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:12.816+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:12.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:12.816+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:14.068 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:14.072+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:14.080 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:14.084+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:14.260 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:13:14.261 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:13:14.267 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:13:14.287 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:13:14.389 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:14.392+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:14.390 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:14.396+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:14.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.804+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:14.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:14.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:14.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:14.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:14.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:14.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.828+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:14.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.828+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:14.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:14.828+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:15.846 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4182509406 2023-10-14T00:13:15.850 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:13:15.852 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242395 2023-10-14T00:13:15.852 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:13:15.852 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4182509406 ]] 2023-10-14T00:13:15.852 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4182509406 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\2\5\0\9\4\0\6 ]] 2023-10-14T00:13:15.853 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:13:16.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:16.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:16.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:16.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:16.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:16.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:16.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:16.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:16.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:16.363+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:16.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:16.363+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:16.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:16.363+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:16.517 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:16.519+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:16.530 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:16.535+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:17.996 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:17.999+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:18.038 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:18.043+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:18.045 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:18.047+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:21.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:21.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:21.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:21.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:21.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:21.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:21.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:21.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:21.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:21.187+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:21.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:21.195+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:21.372 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:21.375+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:21.390 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:21.395+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:23.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:23.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:23.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:23.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:23.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:23.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:23.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:23.691+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:24.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.099+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:24.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:24.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:24.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:24.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:24.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:24.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.151+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:24.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.151+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:24.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.163+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:24.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:24.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:24.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:24.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:24.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.811+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:24.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.811+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:24.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:24.823+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:25.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.119+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:25.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:25.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:25.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:25.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:25.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:25.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.223+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:25.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.223+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:25.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:25.231+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:25.727 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:25.731+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:25.727 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:25.731+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:25.866 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:13:25.868 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:13:25.868 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:13:25.869 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:13:26.161 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:26.167+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:13:26.186 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:26.191+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:26.186 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:26.191+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:26.196 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:26.199+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:26.324 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:26.327+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:26.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:26.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:26.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:26.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:26.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:26.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:26.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.463+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:26.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.487+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:26.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.691+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:26.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:26.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:26.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:26.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:26.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:26.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.727+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:26.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.763+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:26.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:26.779+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:27.604 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4182509406 2023-10-14T00:13:27.621 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:13:27.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242407 2023-10-14T00:13:27.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:13:27.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4182509406 ]] 2023-10-14T00:13:27.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4182509406 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\2\5\0\9\4\0\6 ]] 2023-10-14T00:13:27.624 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:13:28.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:28.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:29.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:29.007+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:31.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:31.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:31.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:31.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:31.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:31.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:31.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:31.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:31.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:31.423+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:31.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:31.423+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:31.639 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:31.643+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:31.661 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:31.663+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:34.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:34.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:34.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:34.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:34.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:34.759+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:37.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:37.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:37.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:37.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:37.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:37.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.155+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:37.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.155+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:37.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:37.163+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:37.638 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:13:37.638 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:13:37.638 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:13:37.639 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:13:38.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:38.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:38.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:38.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:38.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:38.507+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:39.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:39.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:39.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:39.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:39.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:39.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:39.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:39.127+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:39.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:39.127+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:39.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:39.139+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:39.156 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529585824 2023-10-14T00:13:39.158 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:13:39.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242419 2023-10-14T00:13:39.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:13:39.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529585824 ]] 2023-10-14T00:13:39.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529585824 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\2\5\0\9\4\0\6 ]] 2023-10-14T00:13:39.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/1529585824 2023-10-14T00:13:40.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:40.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:40.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:40.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:40.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:40.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.639+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:40.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.639+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:40.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.663+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:40.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.667+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:13:40.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.667+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:13:40.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.695+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:40.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:40.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:40.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:40.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:40.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:40.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.775+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:40.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.775+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:40.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.783+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:40.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:40.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:40.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:40.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:40.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:40.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.863+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:40.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.863+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:40.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:40.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:41.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:13:41.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:13:41.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:41.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:41.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:41.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:41.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:41.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.939+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:41.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.939+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:41.944 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/1529585824 until 2023-10-14T01:13:41.303992+0000 (3600 sec) 2023-10-14T00:13:41.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.987+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:41.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:41.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:42.000 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:13:42.009 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/1529585824 2023-10-14T00:13:42.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:42.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:42.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:42.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:42.890 INFO:tasks.workunit.client.0.smithi183.stderr:listed 13 entries 2023-10-14T00:13:42.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/1529585824 2023-10-14T00:13:42.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:13:44.721 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:44.723+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:45.250 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:45.255+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:45.257 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:45.263+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:45.921 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:45.923+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:46.647 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:46.651+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:49.804 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:49.807+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:49.805 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:49.807+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:49.805 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:49.807+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:50.028 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:50.023+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:50.029 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:50.031+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:50.047 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:50.051+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:51.414 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:51.419+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:51.435 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:51.435+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:51.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:51.959+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:13:51.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:51.959+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:13:51.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:51.963+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:51.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:51.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:51.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:51.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:52.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:52.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:52.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:52.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.019+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:52.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.019+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:52.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.051+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:52.420 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:13:52.423+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:13:52.420 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:13:52.423+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:13:52.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:52.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:52.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:52.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:52.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:52.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:52.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:52.923+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:52.960 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:13:52.961 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:13:52.969 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:13:52.974 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:13:53.454 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:53.459+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:54.494 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529585824 2023-10-14T00:13:54.500 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:13:54.513 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242434 2023-10-14T00:13:54.513 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:13:54.514 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529585824 ]] 2023-10-14T00:13:54.514 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529585824 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\8\5\8\2\4 ]] 2023-10-14T00:13:54.514 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:13:54.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:54.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:54.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:54.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:54.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:54.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:54.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:54.715+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:54.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:54.715+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:54.954 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:54.959+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:54.983 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:54.987+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:55.121 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:55.123+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:55.129 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:55.131+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:13:55.129 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:55.131+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:55.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.886+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:55.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:55.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:55.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:55.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:55.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:55.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.942+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:55.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.942+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:55.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:55.950+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:56.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.318+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:56.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:56.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:56.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:56.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:56.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:56.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.338+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:56.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.338+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:56.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.338+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:56.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:13:56.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:56.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:56.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:56.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:56.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:56.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:56.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:57.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.430+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:13:57.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:57.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:57.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:57.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:13:57.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.502+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:13:57.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.502+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:13:57.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:57.514+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:57.898 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:57.902+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:13:57.910 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:13:57.910+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:13:58.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:58.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:58.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:58.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:58.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:58.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:13:59.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:59.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:13:59.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:59.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:13:59.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:59.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:13:59.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:13:59.082+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:00.006 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.010+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.006 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.010+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.006 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.010+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d28b80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.007 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.010+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb295ce00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.014+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.014+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2e70180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.014+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.014 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.014+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2fbe680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.034 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.038+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.034 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.038+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253d400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.046 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.050+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.046 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.050+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3212180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.046 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.050+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.047 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.050+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325b780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.094 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.098+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.098+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.102+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.102+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325b400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.118+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.118+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.118+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb325b600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.118+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325b980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.130+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.130+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93e80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.137 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.138+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.137 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.138+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325bc80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.158 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.162+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.158 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.162+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325be80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.164 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.166+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.164 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.166+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.179 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.182+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.180 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.182+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.185 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.190+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.186 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.190+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.188 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.190+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.189 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.190+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258e80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.192 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.194+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.193 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.194+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.197 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.198+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.197 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:14:00.198+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:14:00.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.422+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:00.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:00.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:00.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:00.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:00.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.510+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:00.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.510+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:00.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.510+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:00.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.954+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:00.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:00.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:00.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:00.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:00.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:00.998+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:01.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.026+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:01.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:01.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:01.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:01.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.226+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:01.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:01.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:01.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:01.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:01.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:01.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.278+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:01.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.278+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:01.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:01.996 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:01.998+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:14:02.005 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:02.006+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:02.009 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:02.010+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:02.024 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:02.026+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:02.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:02.558+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:02.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:02.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:02.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:02.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:02.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:02.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:02.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:02.582+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:02.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:02.582+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:04.519 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:14:04.519 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:14:04.526 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:14:04.532 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:14:04.627 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:04.630+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:05.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:05.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:05.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:05.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:05.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:05.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:05.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:05.734+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:05.948 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529585824 2023-10-14T00:14:05.955 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:14:05.956 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242445 2023-10-14T00:14:05.956 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:14:05.956 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529585824 ]] 2023-10-14T00:14:05.957 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529585824 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\8\5\8\2\4 ]] 2023-10-14T00:14:05.957 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:14:07.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:07.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:07.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:07.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:07.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:07.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.246+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:07.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.246+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:07.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.274+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:07.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:07.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:07.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:07.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.714+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:07.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.714+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:07.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:07.746+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:08.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:08.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:08.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:08.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:08.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:08.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:08.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:08.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:08.610+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:09.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:09.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:09.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:09.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:09.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:09.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:09.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:09.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:14:09.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:14:09.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:09.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:09.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:09.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:09.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:09.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:09.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.610+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:09.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.610+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:09.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:09.610+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:10.341 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:10.342+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:14:10.355 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:10.358+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:10.372 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:10.374+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:10.373 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:10.374+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:11.106 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:11.110+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:14:11.123 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:11.126+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:11.128 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:11.130+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:11.135 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:11.138+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:11.751 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:11.754+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:14:11.768 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:11.762+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:13.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:13.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:13.081 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:13.078+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:13.087 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:13.090+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:13.087 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:13.090+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:13.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:13.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:13.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:13.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:13.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:13.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:13.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:13.098+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:13.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:13.098+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:13.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:13.098+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:14.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:14.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:14.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:14.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:14.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:14.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:14.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:14.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:14.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:14.702+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:14.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:14.702+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:14.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:14.702+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:14.846 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:14.850+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:15.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.678+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:15.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:15.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:15.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:15.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:15.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:15.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.694+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:15.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.694+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:15.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:15.698+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:15.965 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:14:15.965 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:14:15.969 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:14:15.969 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:14:16.221 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:16.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:16.232 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:16.230+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:16.936 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:14:16.934+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:14:16.937 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:14:16.934+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:14:17.485 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529585824 2023-10-14T00:14:17.494 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:14:17.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242457 2023-10-14T00:14:17.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:14:17.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529585824 ]] 2023-10-14T00:14:17.495 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529585824 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\8\5\8\2\4 ]] 2023-10-14T00:14:17.496 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:14:19.698 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:14:19.702+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:14:19.699 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:14:19.702+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:14:20.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.666+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:14:20.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.678+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:14:20.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:20.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:20.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:20.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:20.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:20.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:20.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:20.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:22.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.242+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:22.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:22.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:22.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:22.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:22.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.266+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:22.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.266+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:22.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:22.302+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:24.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:24.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:24.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:24.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:24.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:24.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:24.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:24.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:24.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:24.054+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:24.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:24.066+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:25.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:25.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:25.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:25.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:25.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.290+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:25.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.290+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:25.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.290+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:25.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:25.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:25.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:25.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:25.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:25.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.834+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:25.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.834+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:25.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:25.850+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:26.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:26.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:26.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:26.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:26.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:26.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.618+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:26.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.618+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:26.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:26.618+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:27.497 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:14:27.502 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:14:27.511 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:14:27.516 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:14:27.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:27.905+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:27.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:27.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:27.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:27.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:27.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:27.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:27.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:27.937+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:28.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:28.569+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:14:28.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:28.569+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:14:28.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:28.569+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:28.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:28.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:28.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:28.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:28.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:28.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:28.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:28.593+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:28.823 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529585824 2023-10-14T00:14:28.823 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:14:28.824 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242468 2023-10-14T00:14:28.824 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:14:28.824 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529585824 ]] 2023-10-14T00:14:28.824 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529585824 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\8\5\8\2\4 ]] 2023-10-14T00:14:28.825 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:14:29.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.437+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:29.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:29.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:29.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:29.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:29.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:29.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.469+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:29.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.469+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:29.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:29.469+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:30.417 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:30.417+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:30.432 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:30.429+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:30.490 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:30.493+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:30.499 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:30.501+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:31.148 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:31.149+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:31.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:31.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:31.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:31.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:31.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:31.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:31.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:31.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:31.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:31.405+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:31.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:31.421+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:34.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.201+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:34.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:34.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:34.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:34.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:34.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.225+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:34.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.225+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:34.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.233+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:34.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:34.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:34.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:34.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:34.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.421+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:34.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:34.421+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:35.097 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:35.097+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:35.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.253+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:35.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:35.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:35.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:35.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:35.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:35.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.261+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:35.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.261+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:35.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.261+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:35.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:35.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:35.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:35.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:35.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:35.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.333+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:35.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:35.337+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:36.970 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:36.973+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:38.829 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:14:38.829 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:14:38.843 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:14:38.856 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:14:39.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:39.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:40.008 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1529585824 2023-10-14T00:14:40.008 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:14:40.009 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242480 2023-10-14T00:14:40.009 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:14:40.010 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1529585824 ]] 2023-10-14T00:14:40.010 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1529585824 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\8\5\8\2\4 ]] 2023-10-14T00:14:40.010 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:14:41.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:41.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:41.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:41.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:41.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:41.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.381+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:41.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.381+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:41.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.413+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:41.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.565+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:41.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:41.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:41.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:41.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:41.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:41.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.609+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:41.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:41.621+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:42.018 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:42.021+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:42.021 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:42.021+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:42.438 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:42.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:42.439 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:42.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:43.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:43.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:43.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:43.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:43.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:43.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.781+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:43.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.781+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:43.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:43.781+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:44.312 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:44.313+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:14:44.313 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:44.313+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:44.517 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:44.517+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:44.538 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:44.541+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:47.303 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:47.305+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:47.310 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:47.313+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:47.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.593+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:47.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:47.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:47.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:47.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:47.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:47.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:47.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:47.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:47.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.705+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:47.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:47.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:47.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:47.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:47.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:47.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.801+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:47.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.801+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:47.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.801+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:47.887 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:47.889+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:47.887 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:47.889+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:47.893 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:47.897+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:47.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.941+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:47.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:47.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:47.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:48.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:48.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:48.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:48.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.057+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:48.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.057+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:48.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.065+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:48.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.157+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:48.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:48.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:48.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:48.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:48.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:48.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.233+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:48.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.233+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:48.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.241+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:48.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:48.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:48.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:48.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:48.801+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:49.223 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:49.225+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:49.236 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:49.237+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:49.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:49.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:49.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.473+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:49.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.473+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:49.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.497+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:49.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:49.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:49.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:49.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:49.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:49.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.997+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:49.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:49.997+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:50.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.025+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:50.029 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:14:50.030 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:14:50.030 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:14:50.042 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:14:50.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.249+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:50.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:50.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:50.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:50.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:50.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:50.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.317+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:50.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.317+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:50.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.317+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:50.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:50.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:50.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:50.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:50.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:50.669+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:51.288 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2475154413 2023-10-14T00:14:51.288 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:14:51.289 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242491 2023-10-14T00:14:51.290 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:14:51.290 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2475154413 ]] 2023-10-14T00:14:51.290 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2475154413 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\5\2\9\5\8\5\8\2\4 ]] 2023-10-14T00:14:51.290 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2475154413 2023-10-14T00:14:52.306 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:52.305+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:14:52.307 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:52.305+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:52.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:52.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:52.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:52.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:52.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.457+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:52.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.457+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:52.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:52.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:52.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:52.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:52.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:52.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:52.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:52.725+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:54.013 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2475154413 until 2023-10-14T01:14:53.366396+0000 (3600 sec) 2023-10-14T00:14:54.095 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2475154413 2023-10-14T00:14:54.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:14:54.253 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:54.253+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:14:54.265 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:54.265+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:14:54.272 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:54.273+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:54.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:54.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:54.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:54.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:54.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:54.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:54.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:54.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:54.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:54.673+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:54.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:54.681+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:55.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.145+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:55.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:55.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:55.180 INFO:tasks.workunit.client.0.smithi183.stderr:listed 14 entries 2023-10-14T00:14:55.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:55.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:55.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:55.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:55.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.217+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:55.219 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2475154413 2023-10-14T00:14:55.220 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:14:55.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.225+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:55.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:55.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:55.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:55.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:55.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:55.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:55.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:55.479 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:14:55.481+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:14:58.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:58.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:58.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:58.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:58.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:58.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:58.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:58.081+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:14:59.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.853+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:14:59.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:14:59.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:14:59.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:14:59.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:14:59.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:14:59.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.929+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:14:59.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.929+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:14:59.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:14:59.937+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:00.011 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.013+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.011 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.013+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2d29580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.020 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.021+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.020 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.021+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.021 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.021+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.021 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.021+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253cd80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.021 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.021+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2931e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.021 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.021+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253de80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.033+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.033+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325ab00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.033+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.033+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325b000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.037+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.037+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325b900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.041+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.041+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffba80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.044 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.045+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.045 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.045+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.048 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.049+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.048 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.049+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.053+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.053+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffbf00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.058 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.057+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.057+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2e71700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.065+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.065+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.065+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.065+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.089+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.089+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.091 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.093+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.091 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.093+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.094 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.097+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.097+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.101+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.101+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259a00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.101+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.101+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:00.109 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:15:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:15:02.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:02.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:03.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:03.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:03.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:03.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.028+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:03.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.028+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:03.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.040+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:03.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:03.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:03.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:03.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:03.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.400+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:03.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.400+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:03.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:03.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.564+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:15:03.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.584+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:03.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:03.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:03.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:03.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:03.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:03.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.632+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:03.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.632+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:03.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:03.648+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:04.179 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:04.180+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:15:04.200 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:04.200+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:04.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:04.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:04.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:04.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:04.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:04.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:04.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:04.716+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:04.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:04.716+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:04.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:04.728+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:05.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:05.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:05.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:05.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:05.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:05.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.064+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:05.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.064+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:05.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.076+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:05.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.152+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:05.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:05.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:05.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:05.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:05.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.204+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:05.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.204+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:05.227 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:15:05.228 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:15:05.239 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:15:05.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:05.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:05.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:05.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:05.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.248+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:05.256 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:15:05.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:05.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:05.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:05.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:05.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.380+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:05.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.392+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:05.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:05.396+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:06.470 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2475154413 2023-10-14T00:15:06.471 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:15:06.471 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242506 2023-10-14T00:15:06.471 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:15:06.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2475154413 ]] 2023-10-14T00:15:06.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2475154413 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\7\5\1\5\4\4\1\3 ]] 2023-10-14T00:15:06.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:15:06.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:06.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:06.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:06.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:06.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:06.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:06.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:06.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:06.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:06.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:06.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:06.940+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:06.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:06.940+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:08.466 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:08.468+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:15:08.477 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:08.476+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:15:08.477 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:08.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:09.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:09.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:09.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:09.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:09.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:09.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:09.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:09.092+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:10.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:10.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:10.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:10.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:10.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.228+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:10.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.228+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:10.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.244+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:10.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:10.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:10.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:10.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:10.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:10.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:10.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:10.788+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:11.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:11.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:11.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:11.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:11.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:11.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.868+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:11.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.868+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:11.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:11.904+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:13.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.696+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:15:13.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.724+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:13.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:13.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:13.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:13.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:13.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:13.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.772+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:13.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.772+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:13.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:13.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:14.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.292+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:14.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:14.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:14.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:14.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:14.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:14.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.388+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:14.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.388+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:14.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:14.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.400+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:14.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:14.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:14.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:14.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:14.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:14.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:16.009 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:16.008+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:15:16.026 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:16.028+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:16.494 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:15:16.494 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:15:16.500 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:15:16.502 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:15:17.886 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2475154413 2023-10-14T00:15:17.886 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:15:17.957 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242517 2023-10-14T00:15:17.958 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:15:17.958 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2475154413 ]] 2023-10-14T00:15:17.958 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2475154413 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\7\5\1\5\4\4\1\3 ]] 2023-10-14T00:15:17.958 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:15:18.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:18.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:18.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:18.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:18.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:18.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:18.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:18.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:18.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:18.980+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:19.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:19.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:19.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:19.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:19.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:19.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.544+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:19.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.544+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:19.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:19.548+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:19.846 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:15:19.848+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:15:20.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:20.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:20.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:20.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:20.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:20.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:20.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:20.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:20.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:20.828+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:20.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:20.828+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:20.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:20.848+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:22.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:22.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:22.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:22.316+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:23.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.380+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:15:23.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.380+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:15:23.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.388+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:23.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:23.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:23.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:23.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:23.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:23.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.436+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:23.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.436+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:23.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.468+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:23.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:23.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:23.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:23.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:23.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:23.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.556+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:23.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.556+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:23.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.560+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:23.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.692+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:23.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:23.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:23.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:23.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:23.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:23.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.780+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:23.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.780+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:23.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:23.792+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:26.953 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:15:26.952+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:15:27.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.204+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:27.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:27.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:27.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:27.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:27.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:27.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.240+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:27.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.240+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:27.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.248+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:27.937 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:27.940+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:15:27.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:27.948+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:27.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:27.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:27.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:27.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:27.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:27.992+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:28.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:28.008+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:28.033 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:15:28.038 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:15:28.039 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:15:28.044 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:15:29.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:29.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:29.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:29.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:29.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.512+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:29.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.512+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:29.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.540+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:29.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:29.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:29.586 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2475154413 2023-10-14T00:15:29.586 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:15:29.589 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242529 2023-10-14T00:15:29.589 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:15:29.590 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2475154413 ]] 2023-10-14T00:15:29.590 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2475154413 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\7\5\1\5\4\4\1\3 ]] 2023-10-14T00:15:29.590 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:15:29.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.956+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:29.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:29.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:29.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:29.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:29.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:29.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.992+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:29.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.992+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:29.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:29.992+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:31.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:31.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:31.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:31.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:31.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:31.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:31.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:31.116+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:33.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.680+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:33.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:33.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:33.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:33.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:33.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:33.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.716+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:33.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.716+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:33.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:33.716+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:34.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.644+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:34.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:34.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:34.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:34.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:34.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:34.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.688+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:34.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.688+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:34.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:34.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:36.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:36.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:36.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:36.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:36.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:36.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:36.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:36.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:36.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:36.436+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:36.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:36.436+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:36.477 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:36.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:39.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:39.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:39.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:39.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:39.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:39.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.379+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:39.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.379+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:39.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.383+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:39.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:39.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:39.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:39.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:39.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.455+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:39.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:39.471+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:39.618 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:15:39.618 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:15:39.619 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:15:39.621 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:15:40.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.195+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:15:40.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.219+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:15:40.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.243+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:40.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:40.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:40.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:40.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:40.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:40.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.271+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:40.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.271+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:40.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.331+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:40.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.711+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:40.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:40.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:40.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:40.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:40.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:40.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.747+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:40.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.747+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:40.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:40.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:41.019 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:41.023+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:41.222 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2475154413 2023-10-14T00:15:41.223 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:15:41.229 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242541 2023-10-14T00:15:41.230 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:15:41.230 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2475154413 ]] 2023-10-14T00:15:41.230 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2475154413 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\7\5\1\5\4\4\1\3 ]] 2023-10-14T00:15:41.231 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:15:42.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.399+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:15:42.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.411+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:15:42.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.427+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:42.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:42.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:42.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:42.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:42.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:42.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.435+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:42.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.483+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:42.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.587+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:42.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:42.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:42.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:42.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:42.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:42.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.659+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:42.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.659+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:42.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.659+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:42.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:42.943+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:43.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.651+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:43.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:43.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:43.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:43.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:43.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:43.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.691+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:43.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.707+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:43.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:43.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:43.741 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:43.743+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:15:43.749 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:43.751+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:43.788 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:43.791+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:15:43.846 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:43.851+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:44.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:44.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:45.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:45.903+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:46.326 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:46.331+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:15:46.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:46.363+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:15:46.367 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:46.371+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:15:46.367 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:46.371+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:46.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:46.847+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:46.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:46.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:46.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:46.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:46.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:46.899+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:47.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:47.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:47.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:47.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:47.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.695+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:47.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.695+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:47.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:47.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:47.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:47.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:47.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:47.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.899+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:47.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.899+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:47.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:47.931+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:48.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:48.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:48.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:48.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:48.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:48.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.523+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:48.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.523+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:48.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:48.531+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:48.777 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:48.779+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:15:48.777 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:48.779+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:49.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.091+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:49.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:49.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:49.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:49.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:49.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.183+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:49.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.183+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:49.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.183+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:49.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.183+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:49.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:49.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:49.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:49.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:49.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.203+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:49.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.203+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:49.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:49.203+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:49.414 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:15:49.415+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:15:49.414 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:15:49.415+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:15:50.804 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:50.807+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:15:51.281 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:15:51.283 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:15:51.284 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:15:51.293 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:15:52.688 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2475154413 2023-10-14T00:15:52.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:52.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:52.698 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:15:52.699 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242552 2023-10-14T00:15:52.699 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:15:52.700 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2475154413 ]] 2023-10-14T00:15:52.700 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2475154413 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\7\5\1\5\4\4\1\3 ]] 2023-10-14T00:15:52.700 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:15:52.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:52.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:52.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:52.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:52.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:52.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:52.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:52.775+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:52.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:52.775+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:52.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:52.775+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:54.602 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:54.607+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:15:54.642 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:54.647+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:54.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:54.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:54.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:54.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:54.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:54.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:54.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:54.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:54.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:54.679+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:54.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:54.679+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:54.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:54.679+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:54.996 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:54.999+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:57.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:57.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:57.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:57.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:57.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:57.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:57.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:57.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:15:57.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:57.359+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:15:57.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:57.359+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:15:57.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:57.359+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:58.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:58.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:58.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:58.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:15:58.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:58.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:58.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:58.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:58.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:58.051+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:15:58.981 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:58.983+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:15:58.984 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:15:58.987+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:15:59.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:59.399+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:15:59.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:59.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:15:59.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:59.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:15:59.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:59.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:15:59.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:15:59.423+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:01.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:01.123+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:01.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:01.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:01.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:01.155+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:02.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:02.159+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:02.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:02.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:02.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:02.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:02.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:02.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:02.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:02.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:02.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:02.231+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:02.702 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:16:02.703 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:16:02.703 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:16:02.703 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:16:03.583 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:03.587+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:16:03.584 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:03.587+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:03.632 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1395831165 2023-10-14T00:16:03.632 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:16:03.633 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242563 2023-10-14T00:16:03.634 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:16:03.634 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1395831165 ]] 2023-10-14T00:16:03.634 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1395831165 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\7\5\1\5\4\4\1\3 ]] 2023-10-14T00:16:03.634 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/1395831165 2023-10-14T00:16:04.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:04.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:04.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:04.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:04.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:04.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:04.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:04.555+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:04.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:04.555+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:05.049 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.051+0000 7f660e1ef640 -1 librbd::ImageWatcher: 0x55ccb2b58900 image watch failed: 94337648799744, (107) Transport endpoint is not connected 2023-10-14T00:16:05.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.051+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb2b58900 handle_error: handle=94337648799744: (107) Transport endpoint is not connected 2023-10-14T00:16:05.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.051+0000 7f660e1ef640 -1 librbd::watcher::RewatchRequest: 0x55ccb2a097c0 handle_unwatch client blocklisted 2023-10-14T00:16:05.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.051+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb2b58900 handle_rewatch: client blocklisted 2023-10-14T00:16:05.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.123+0000 7f660d9ee640 -1 librbd::ImageWatcher: 0x55ccb2b58c00 image watch failed: 94337659181056, (107) Transport endpoint is not connected 2023-10-14T00:16:05.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.123+0000 7f660d9ee640 -1 librbd::Watcher: 0x55ccb2b58c00 handle_error: handle=94337659181056: (107) Transport endpoint is not connected 2023-10-14T00:16:05.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.123+0000 7f660e1ef640 -1 librbd::ImageWatcher: 0x55ccb2b58f00 image watch failed: 94337659175936, (107) Transport endpoint is not connected 2023-10-14T00:16:05.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.123+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb2b58f00 handle_error: handle=94337659175936: (107) Transport endpoint is not connected 2023-10-14T00:16:05.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.127+0000 7f660e1ef640 -1 librbd::watcher::RewatchRequest: 0x55ccb2b3c5f0 handle_unwatch client blocklisted 2023-10-14T00:16:05.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.127+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb2b58c00 handle_rewatch: client blocklisted 2023-10-14T00:16:05.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.127+0000 7f660e1ef640 -1 librbd::watcher::RewatchRequest: 0x55ccb2ad49b0 handle_unwatch client blocklisted 2023-10-14T00:16:05.128 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:05.127+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb2b58f00 handle_rewatch: client blocklisted 2023-10-14T00:16:06.044 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/1395831165 until 2023-10-14T01:16:05.248694+0000 (3600 sec) 2023-10-14T00:16:06.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.063+0000 7f660e1ef640 -1 librbd::SnapshotCreateRequest: failed to notify unquiesce: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.063+0000 7f660e1ef640 -1 librbd::mirror::snapshot::CreatePrimaryRequest: 0x55ccb3211040 handle_create_snapshot: failed to create mirror snapshot: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.063+0000 7f660e1ef640 -1 librbd::io::AioCompletion: 0x55ccb34d0840 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.063+0000 7f660e1ef640 -1 librbd::image::RefreshRequest: failed to retrieve mutable metadata: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.063+0000 7f660e1ef640 -1 librbd::io::AioCompletion: 0x55ccb34a3ce0 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.063+0000 7f660d9ee640 -1 librbd::exclusive_lock::PreReleaseRequest: 0x55ccb34ac440 handle_set_require_lock: failed to set lock: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.067+0000 7f660d9ee640 -1 librbd::managed_lock::ReleaseRequest: 0x55ccb29fb6e0 handle_unlock: failed to unlock: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.067+0000 7f660d9ee640 -1 librbd::SnapshotCreateRequest: failed to notify unquiesce: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.067+0000 7f660d9ee640 -1 librbd::mirror::snapshot::CreatePrimaryRequest: 0x55ccb3210340 handle_create_snapshot: failed to create mirror snapshot: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.067+0000 7f660d9ee640 -1 librbd::io::AioCompletion: 0x55ccb34a29a0 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.067+0000 7f660d9ee640 -1 librbd::SnapshotCreateRequest: failed to notify unquiesce: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.067+0000 7f660d9ee640 -1 librbd::mirror::snapshot::CreatePrimaryRequest: 0x55ccb3211110 handle_create_snapshot: failed to create mirror snapshot: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.067+0000 7f660d9ee640 -1 librbd::io::AioCompletion: 0x55ccb34454a0 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660e1ef640 -1 librbd::image::RefreshRequest: failed to retrieve mutable metadata: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660e1ef640 -1 librbd::io::AioCompletion: 0x55ccb34a3ce0 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660e1ef640 -1 librbd::exclusive_lock::PreReleaseRequest: 0x55ccb3382bc0 handle_set_require_lock: failed to set lock: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660e1ef640 -1 librbd::image::RefreshRequest: failed to retrieve mutable metadata: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660e1ef640 -1 librbd::io::AioCompletion: 0x55ccb34a29a0 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.068 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660e1ef640 -1 librbd::exclusive_lock::PreReleaseRequest: 0x55ccb343fdc0 handle_set_require_lock: failed to set lock: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.068 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660d9ee640 -1 librbd::managed_lock::ReleaseRequest: 0x55ccb259ade0 handle_unlock: failed to unlock: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.068 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:16:06.071+0000 7f660d9ee640 -1 librbd::managed_lock::ReleaseRequest: 0x55ccb2c59b60 handle_unlock: failed to unlock: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:16:06.079 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:16:06.082 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/1395831165 2023-10-14T00:16:07.073 INFO:tasks.workunit.client.0.smithi183.stderr:listed 15 entries 2023-10-14T00:16:07.105 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/1395831165 2023-10-14T00:16:07.105 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:16:09.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:09.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:09.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:09.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:09.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:09.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:09.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:09.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:09.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:09.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:09.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:09.615+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:09.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:09.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:10.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.567+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:10.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:10.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:10.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:10.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:10.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.619+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:10.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.619+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:10.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.627+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:10.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:10.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:10.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.671+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:10.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:10.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:16:10.872 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:10.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:16:10.872 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:10.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:16:10.872 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:10.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:10.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:10.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:10.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:10.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.983+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:10.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:10.983+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:11.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.023+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:11.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:11.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:11.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.083+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:11.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:11.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:11.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:11.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:11.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.627+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:11.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.627+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:11.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:11.627+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:12.177 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:12.179+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:16:12.179 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:12.183+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:12.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:12.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:12.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:12.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:12.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:12.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:12.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:12.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.463+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:12.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:12.467+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:14.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.355+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:16:14.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.355+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:16:14.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.355+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:14.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:14.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:14.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:14.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:14.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:14.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.411+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:14.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.411+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:14.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:14.411+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:15.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:15.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:15.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:15.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:15.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:15.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.291+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:15.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.291+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:15.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:15.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.638+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:15.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:15.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:15.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:15.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:15.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:15.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.714+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:15.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.714+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:15.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:15.722+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:16.793 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:16.794+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:16:16.803 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:16.806+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:17.118 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:16:17.121 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:16:17.121 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:16:17.123 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:16:18.563 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1395831165 2023-10-14T00:16:18.570 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:16:18.582 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242578 2023-10-14T00:16:18.582 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:16:18.583 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1395831165 ]] 2023-10-14T00:16:18.583 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1395831165 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\3\9\5\8\3\1\1\6\5 ]] 2023-10-14T00:16:18.583 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:16:19.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.926+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:19.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:19.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:19.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:19.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:19.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:19.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.962+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:19.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.962+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:19.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:19.962+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:21.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:21.814+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:16:21.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:21.814+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:21.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:21.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:21.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:21.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:21.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:21.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:21.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:21.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:21.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:21.906+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:26.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:26.982+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:26.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:26.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:27.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:27.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:27.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:27.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.042+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:27.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.042+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:27.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.042+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:27.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.390+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:27.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:27.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:27.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:27.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:27.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:27.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.414+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:27.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.414+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:27.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.414+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:27.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:27.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:27.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:27.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:27.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.582+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:27.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.582+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:27.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:27.606+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:28.587 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:16:28.588 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:16:28.594 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:16:28.595 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:16:29.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:29.514+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:16:29.554 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:29.558+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:30.150 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1395831165 2023-10-14T00:16:30.151 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:16:30.152 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242590 2023-10-14T00:16:30.152 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:16:30.152 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1395831165 ]] 2023-10-14T00:16:30.152 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1395831165 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\3\9\5\8\3\1\1\6\5 ]] 2023-10-14T00:16:30.153 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:16:30.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.350+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:30.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:30.384 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:30.386+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:16:30.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:30.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:30.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:30.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.402+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:30.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.402+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:30.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:30.406+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:35.807 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:35.810+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:36.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.638+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:16:36.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.638+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:16:36.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.646+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:36.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:36.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:36.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:36.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:36.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:36.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.690+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:36.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.690+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:36.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:36.690+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:39.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.342+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:39.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:39.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:39.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:39.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:39.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:39.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.374+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:39.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.374+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:39.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.374+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:39.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.442+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:39.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:39.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:39.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:39.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:39.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:39.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:39.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:39.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:39.474+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:39.480 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:39.482+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:40.164 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:16:40.164 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:16:40.164 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:16:40.164 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:16:40.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:40.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:40.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:40.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:40.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:40.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.450+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:40.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.450+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:40.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:40.462+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:41.584 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1395831165 2023-10-14T00:16:41.584 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:16:41.585 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242601 2023-10-14T00:16:41.585 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:16:41.585 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1395831165 ]] 2023-10-14T00:16:41.586 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1395831165 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\3\9\5\8\3\1\1\6\5 ]] 2023-10-14T00:16:41.586 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:16:41.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.630+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:41.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:41.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:41.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:41.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:41.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:41.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:41.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:41.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:41.678+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:42.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:42.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:42.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:42.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:42.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:42.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:42.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:42.430+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:42.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:42.430+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:42.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:42.454+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:42.696 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:42.698+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:43.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:43.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:43.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:43.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:43.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:43.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:43.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:43.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.802+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:43.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.802+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:43.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:43.802+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:44.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:44.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:44.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:44.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:45.091 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:45.090+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:16:45.107 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:45.110+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:45.867 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:45.870+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:16:45.876 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:45.878+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:46.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:46.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:46.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:46.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:46.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.670+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:46.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.670+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:46.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.714+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:46.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:46.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:46.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:46.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:46.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.802+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:46.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.802+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:46.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:46.810+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:48.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:48.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:48.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:48.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:48.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:48.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:48.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:48.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:48.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:48.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:48.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:48.270+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:49.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.738+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:16:49.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.738+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:49.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:49.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:49.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:49.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:49.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:49.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.810+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:49.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.810+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:49.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:49.830+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:50.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.206+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:16:50.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.214+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:16:50.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.218+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:50.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:50.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:50.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:50.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:50.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:50.266+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:50.739 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:50.742+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:16:51.589 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:16:51.590 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:16:51.591 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:16:51.594 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:16:52.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:52.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:52.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:52.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:52.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:52.766+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:52.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1395831165 2023-10-14T00:16:52.970 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:16:52.971 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242612 2023-10-14T00:16:52.971 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:16:52.972 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1395831165 ]] 2023-10-14T00:16:52.972 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1395831165 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\3\9\5\8\3\1\1\6\5 ]] 2023-10-14T00:16:52.972 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:16:54.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.754+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:54.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:54.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:54.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:54.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:54.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.810+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:54.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.810+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:54.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:54.814+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:58.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.049+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:58.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:58.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:58.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:58.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:58.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.093+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:58.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.093+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:58.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:58.093+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:59.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.821+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:16:59.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:16:59.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:16:59.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:16:59.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:16:59.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:16:59.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.885+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:16:59.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.885+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:16:59.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:16:59.885+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:16:59.926 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:16:59.925+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:02.979 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:17:02.989 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:17:02.989 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:17:03.020 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:17:04.279 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1395831165 2023-10-14T00:17:04.283 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:17:04.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242624 2023-10-14T00:17:04.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:17:04.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1395831165 ]] 2023-10-14T00:17:04.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1395831165 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\3\9\5\8\3\1\1\6\5 ]] 2023-10-14T00:17:04.285 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:17:04.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:04.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:04.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:04.585+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:04.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:04.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:04.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:04.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:04.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:04.601+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:04.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:04.601+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:04.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:04.613+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:06.223 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:06.225+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:17:06.264 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:06.265+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:06.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:06.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:06.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:06.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:06.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.485+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:06.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.485+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:06.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:06.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.821+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:06.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:06.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:06.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:06.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:06.853+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:09.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:09.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:09.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:09.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:09.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.597+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:09.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.613+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:09.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.613+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:09.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:09.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:09.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:09.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.709+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:09.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:09.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:09.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:10.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:10.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:10.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:10.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:10.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:10.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:10.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:10.325+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:10.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:10.325+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:10.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:10.409+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:11.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:11.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:11.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:11.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:11.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:11.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:11.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:11.137+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:14.288 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:17:14.289 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:17:14.310 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:17:14.310 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:17:15.580 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:15.581+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:15.602 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:15.601+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:15.763 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4188019524 2023-10-14T00:17:15.770 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:17:15.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242635 2023-10-14T00:17:15.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:17:15.781 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4188019524 ]] 2023-10-14T00:17:15.781 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4188019524 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\3\9\5\8\3\1\1\6\5 ]] 2023-10-14T00:17:15.781 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/4188019524 2023-10-14T00:17:17.471 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:17.469+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:17.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.625+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:17.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:17.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:17.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:17.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:17.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.657+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:17.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.657+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:17.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:17.657+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:18.069 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/4188019524 until 2023-10-14T01:17:17.375910+0000 (3600 sec) 2023-10-14T00:17:18.113 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:17:18.113 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/4188019524 2023-10-14T00:17:18.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.709+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:18.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:18.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:18.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:18.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:18.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:18.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.753+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:18.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:18.769+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:19.210 INFO:tasks.workunit.client.0.smithi183.stderr:listed 16 entries 2023-10-14T00:17:19.258 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/4188019524 2023-10-14T00:17:19.258 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:17:19.849 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:19.849+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:17:19.871 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:19.873+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:20.351 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:20.353+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:20.619 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:20.621+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:17:20.643 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:20.641+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:20.651 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:20.653+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:20.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:20.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:20.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:20.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:20.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:20.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.729+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:20.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.729+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:20.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:20.741+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:21.524 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:21.525+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:17:21.539 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:21.541+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:21.547 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:21.549+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:23.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:23.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:23.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:23.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:23.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:23.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.793+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:23.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.793+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:23.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:23.801+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:24.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.385+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:24.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:24.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:24.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:24.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:24.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:24.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:24.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:24.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:24.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:24.742 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:24.741+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:26.235 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:26.237+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:17:26.244 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:26.245+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:26.272 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:26.273+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:26.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.293+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:26.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:26.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:26.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:26.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:26.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:26.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:26.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.625+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:26.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:26.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:26.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:26.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:26.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:26.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.733+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:26.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.733+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:26.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:26.737+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:27.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:27.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:27.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:27.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:27.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:27.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:27.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:27.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:27.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:27.057+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:27.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:27.057+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:27.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:27.057+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:28.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.333+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:28.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:28.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:28.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:28.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:28.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:28.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:28.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:28.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:28.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.441+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:17:28.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.441+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:17:28.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.465+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:28.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:28.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:28.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:28.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:28.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:28.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.501+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:28.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.501+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:28.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.521+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:28.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.525+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:17:28.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:28.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:28.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:28.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:28.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:28.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.573+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:28.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:28.585+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:29.260 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:17:29.260 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:17:29.266 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:17:29.287 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:17:29.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.721+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:29.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:29.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:29.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:29.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:29.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:29.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.773+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:29.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.773+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:29.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:29.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:30.914 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4188019524 2023-10-14T00:17:30.922 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:17:30.923 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242650 2023-10-14T00:17:30.923 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:17:30.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4188019524 ]] 2023-10-14T00:17:30.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4188019524 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\8\0\1\9\5\2\4 ]] 2023-10-14T00:17:30.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:17:31.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.289+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:31.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:31.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:31.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:31.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:31.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:31.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:31.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:31.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:31.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:31.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:31.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:31.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:31.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.921+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:31.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.921+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:31.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:31.937+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:33.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:33.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:33.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:33.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:33.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:33.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:33.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:33.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:33.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:33.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:33.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:33.513+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:34.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.349+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:34.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:34.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:34.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:34.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:34.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:34.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.429+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:34.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.429+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:34.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:34.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:34.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:34.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:34.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.493+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:34.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:34.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:34.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:34.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.549+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:34.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:34.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:34.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:34.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.753+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:34.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.753+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:34.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:34.765+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:35.810 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:35.813+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:35.819 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:35.821+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:36.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.085+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:17:36.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.085+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:17:36.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.085+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:36.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:36.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:36.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:36.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:36.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:36.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.113+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:36.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.113+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:36.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:36.125+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:37.562 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:37.561+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:37.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:37.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:37.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:37.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:37.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:37.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:37.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:37.981+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:38.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:38.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:38.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:38.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:38.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:38.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:38.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:38.277+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:38.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:38.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:38.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:38.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:39.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.921+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:39.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:39.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:39.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:39.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:40.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:40.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.973+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:40.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.973+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:40.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:39.973+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:40.949 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:17:40.950 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:17:40.957 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:17:40.958 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:17:41.682 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:41.685+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:41.682 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:41.685+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:42.323 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4188019524 2023-10-14T00:17:42.328 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:17:42.332 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242662 2023-10-14T00:17:42.332 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:17:42.332 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4188019524 ]] 2023-10-14T00:17:42.333 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4188019524 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\8\0\1\9\5\2\4 ]] 2023-10-14T00:17:42.333 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:17:42.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.537+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:42.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:42.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:42.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:42.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:42.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:42.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.557+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:42.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.557+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:42.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.565+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:42.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.769+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:42.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:42.806 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:42.805+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:42.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:42.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:42.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:42.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:42.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.857+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:42.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.857+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:42.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:42.857+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:43.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:43.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:43.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:43.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:43.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:43.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:43.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:43.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:43.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:43.101+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:43.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:43.101+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:43.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:43.101+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:44.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:44.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:44.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:44.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:44.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:44.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.724+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:44.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.724+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:44.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:44.736+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:47.288 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:47.288+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:47.289 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:47.288+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:49.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:49.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:49.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:49.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:49.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:49.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:49.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:49.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:49.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.516+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:49.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.516+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:49.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:49.520+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:50.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:50.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:50.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:50.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:50.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.108+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:50.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.108+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:50.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.120+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:50.157 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:50.156+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:50.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:50.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:50.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:50.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:50.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:50.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:50.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:50.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:50.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.576+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:50.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:50.576+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:51.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:51.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:51.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:51.168+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:52.341 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:17:52.348 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:17:52.355 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:17:52.369 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:17:52.606 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:52.604+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:52.607 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:52.608+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:52.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:52.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:52.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:52.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:52.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:52.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.872+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:52.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.872+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:52.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:52.876+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:53.424 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:53.424+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:17:53.425 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:53.428+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:53.430 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:53.432+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:53.817 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4188019524 2023-10-14T00:17:53.819 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:17:53.820 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242673 2023-10-14T00:17:53.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:17:53.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4188019524 ]] 2023-10-14T00:17:53.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4188019524 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\8\0\1\9\5\2\4 ]] 2023-10-14T00:17:53.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:17:54.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:54.664+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:54.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:54.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:54.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:54.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:54.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:54.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:54.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:54.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:54.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:54.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:54.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:54.700+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:55.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:55.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:55.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:55.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:55.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.520+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:55.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.520+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:55.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.544+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:55.785 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:55.784+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:55.801 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:55.800+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:55.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:55.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:55.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:55.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:55.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:55.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.980+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:55.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.980+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:55.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:55.980+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:56.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:56.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:56.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.288+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:56.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.420+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:17:56.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.432+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:17:56.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.440+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:56.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:56.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:56.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:56.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:56.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:56.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.480+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:56.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.480+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:56.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.512+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:56.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.596+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:17:56.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.596+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:17:56.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.596+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:17:56.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:17:56.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:56.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:17:56.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.616+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:56.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.616+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:56.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.616+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:17:56.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.616+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:17:56.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:56.616+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:17:56.726 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:56.728+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:17:56.726 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:17:56.728+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:17:58.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:58.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:17:58.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:58.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:17:58.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:58.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:17:58.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:17:58.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.024+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.024+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.024+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.024+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3178b80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.036 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.036+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.036 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.036+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e4000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.036+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.036+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3178200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.044+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.044+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e5680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.044 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.044+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.044 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.044+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3492500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.068 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.068 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3492580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.069 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.069 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3492f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.069 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3492900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3492a00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3492c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.071 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.068+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e5800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.088+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.088+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.088+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.088+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.088+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.088+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.092+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.092+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.092+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.092+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493a00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.116+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.116+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.116+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.116+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.116+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:18:00.116+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332e000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:18:02.595 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:02.596+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:03.826 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:18:03.826 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:18:03.830 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:18:03.839 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:18:03.992 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:03.988+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:04.013 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:04.016+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:05.171 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4188019524 2023-10-14T00:18:05.171 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:18:05.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242685 2023-10-14T00:18:05.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:18:05.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4188019524 ]] 2023-10-14T00:18:05.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4188019524 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\8\0\1\9\5\2\4 ]] 2023-10-14T00:18:05.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:18:05.334 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:05.336+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:05.335 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:05.336+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:06.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.208+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:06.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:06.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:06.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:06.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:06.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.240+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:06.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.240+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:06.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.240+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:06.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:06.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:06.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:06.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:06.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.680+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:06.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.680+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:06.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.684+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:06.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:06.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:06.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:06.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:06.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:06.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:06.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.000+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:06.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.000+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:07.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.000+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:07.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:07.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:07.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:07.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:07.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:07.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.124+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:07.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.124+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:07.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.136+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:07.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:07.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:07.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:08.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:08.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:08.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:08.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:08.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:08.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:08.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:08.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:08.396+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:08.706 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:08.700+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:09.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:09.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:09.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:09.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:09.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.636+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:09.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.636+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:09.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.648+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:09.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.652+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:09.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:09.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:09.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:09.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:09.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:09.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:09.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:09.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.796+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:09.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.796+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:09.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:09.808+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:10.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:10.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:10.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:10.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:10.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:10.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:10.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:10.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:10.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:10.504+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:10.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:10.504+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:10.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:10.504+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:11.406 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:11.408+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:11.406 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:11.408+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:11.573 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:11.572+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:12.180 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:12.180+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:18:12.185 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:12.184+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:12.209 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:12.208+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:12.209 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:12.208+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:12.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.660+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:12.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:12.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:12.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:12.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:12.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:12.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.736+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:12.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.736+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:12.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.760+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:12.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.796+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:12.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:12.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:12.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:12.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:12.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:12.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.864+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:12.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.864+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:12.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:12.864+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:13.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.784+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:13.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:13.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:13.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:13.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:13.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:13.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.848+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:13.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.848+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:13.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:13.848+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:14.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:14.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:14.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:14.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:14.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:14.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.780+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:14.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.780+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:14.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:14.780+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:15.181 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:18:15.181 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:18:15.182 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:18:15.187 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:18:16.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.204+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:16.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:16.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:16.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:16.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:16.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:16.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:16.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.268+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:16.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:16.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:16.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:16.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:16.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:16.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:16.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.388+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:16.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.388+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:16.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.388+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:16.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4188019524 2023-10-14T00:18:16.661 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:18:16.668 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242696 2023-10-14T00:18:16.669 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:18:16.669 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4188019524 ]] 2023-10-14T00:18:16.669 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4188019524 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\8\0\1\9\5\2\4 ]] 2023-10-14T00:18:16.669 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:18:16.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:16.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:16.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:16.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:16.976+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:17.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:17.004+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:18.018 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:18.020+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:18.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:18.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:18.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:18.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:18.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:18.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:18.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:18.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:18.268+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:19.057 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:19.056+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:19.077 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:19.080+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:19.095 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:19.096+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:19.095 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:19.096+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:19.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:19.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:19.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:19.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:19.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:19.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.160+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:19.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.160+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:19.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:19.180+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:20.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:20.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:20.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:20.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:20.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:20.680+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:20.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:20.680+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:21.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.596+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:18:21.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.596+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:18:21.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.600+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:21.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:21.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:21.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:21.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:21.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:21.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.648+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:21.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.648+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:21.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.648+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:21.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.800+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:21.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:21.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:21.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:21.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:21.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.828+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:21.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.828+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:21.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:21.828+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:22.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.220+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:18:22.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.248+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:18:22.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.248+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:22.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:22.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:22.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:22.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:22.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:22.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.260+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:22.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.260+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:22.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.260+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:22.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.948+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:22.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:22.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:22.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:22.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:22.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:22.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.960+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:22.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:22.960+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:23.919 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:23.924+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:23.963 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:23.968+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:24.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:24.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:24.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:24.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:24.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:24.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:24.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:24.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:25.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:25.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:25.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:25.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:25.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:25.804+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:25.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:25.804+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:26.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:26.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:26.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:26.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:26.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:26.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.504+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:26.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.512+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:26.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:26.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:26.671 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:18:26.671 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:18:26.673 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:18:26.692 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:18:27.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.528+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:27.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:27.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:27.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:27.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:27.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:27.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.584+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:27.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.584+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:27.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:27.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:28.322 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1180091150 2023-10-14T00:18:28.323 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:18:28.324 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242708 2023-10-14T00:18:28.324 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:18:28.324 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1180091150 ]] 2023-10-14T00:18:28.324 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1180091150 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\8\8\0\1\9\5\2\4 ]] 2023-10-14T00:18:28.325 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/1180091150 2023-10-14T00:18:28.501 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:28.508+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:28.522 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:28.528+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:28.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:28.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:28.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:28.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:28.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:28.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:28.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:28.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:28.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:28.728+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:28.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:28.728+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:28.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:28.728+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:30.442 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:30.448+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:30.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:30.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:30.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:30.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:30.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:30.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:30.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:30.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:30.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:30.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:30.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:30.556+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:30.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:30.556+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:31.098 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/1180091150 until 2023-10-14T01:18:30.467988+0000 (3600 sec) 2023-10-14T00:18:31.157 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/1180091150 2023-10-14T00:18:31.157 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:18:31.403 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:31.408+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:31.415 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:31.420+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:31.462 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:31.468+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:31.496 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:31.500+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:18:31.503 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:31.508+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:31.508 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:18:31.512+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:18:31.517 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:31.520+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:31.534 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:31.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:32.222 INFO:tasks.workunit.client.0.smithi183.stderr:listed 17 entries 2023-10-14T00:18:32.261 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/1180091150 2023-10-14T00:18:32.261 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:18:32.916 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:32.920+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:33.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:33.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:33.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:33.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:33.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:33.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:33.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:33.423+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:33.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:33.423+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:35.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:35.311+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:35.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:35.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:35.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:35.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:35.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:35.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:35.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:35.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:35.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:35.359+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:36.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:36.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:36.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:36.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:36.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:36.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.395+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:36.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.395+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:36.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:36.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:36.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:36.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:36.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.779+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:36.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:36.779+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:37.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.155+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:37.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:37.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:37.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:37.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:37.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:37.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.183+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:37.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.183+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:37.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.183+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:37.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.495+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:18:37.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.495+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:18:37.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.495+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:37.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:37.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:37.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:37.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:37.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:37.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.547+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:37.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.547+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:37.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:37.551+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:37.952 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:37.955+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:38.411 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:38.415+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:38.450 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:38.455+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:39.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.023+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:39.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:39.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:39.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:39.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:39.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:39.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:39.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:39.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:39.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:39.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:39.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.083+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:39.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.651+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:39.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:39.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:39.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:39.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:39.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:39.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.679+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:39.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.679+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:39.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:39.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.799+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:39.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:39.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:39.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:39.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:39.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:39.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.823+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:39.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.823+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:39.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.827+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:39.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.923+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:39.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:39.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:39.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:39.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:39.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:39.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.979+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:39.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.979+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:39.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:39.979+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:40.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:40.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:40.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:40.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:40.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:40.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:40.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:40.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:40.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:40.447+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:40.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:40.447+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:40.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:40.451+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:41.430 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:41.435+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:41.436 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:41.439+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:41.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.891+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:41.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:41.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:41.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:41.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:41.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:41.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.959+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:41.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.959+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:41.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:41.959+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:42.270 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:18:42.273 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:18:42.281 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:18:42.299 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:18:42.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:42.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:42.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:42.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:42.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:42.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:42.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:42.355+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:43.654 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1180091150 2023-10-14T00:18:43.654 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:18:43.655 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242723 2023-10-14T00:18:43.655 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:18:43.655 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1180091150 ]] 2023-10-14T00:18:43.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1180091150 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\1\8\0\0\9\1\1\5\0 ]] 2023-10-14T00:18:43.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:18:43.788 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:18:43.791+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:18:44.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.007+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:18:44.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.011+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:44.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:44.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:44.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:44.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:44.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:44.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.087+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:44.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.087+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:44.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.107+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:44.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:44.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:44.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:44.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:44.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.763+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:44.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:44.803+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:46.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:46.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:46.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:46.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:46.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:46.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:46.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:46.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:46.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:46.903+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:46.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:46.927+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:46.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:46.927+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:48.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:47.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:48.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:47.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:48.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:47.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:48.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:48.011+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:48.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:48.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:48.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:48.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:48.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:48.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:48.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:48.595+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:48.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:48.595+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:48.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:48.611+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:49.345 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:49.347+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:49.386 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:49.391+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:49.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.911+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:49.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:49.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:49.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:49.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:49.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:49.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.983+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:49.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.983+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:49.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:49.987+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:50.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:50.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:50.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:50.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:50.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:50.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:50.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:50.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:50.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:50.771+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:50.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:50.771+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:50.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:50.815+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:51.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.703+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:51.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:51.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:51.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:51.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:51.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.715+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:51.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.715+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:51.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.715+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:51.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.835+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:51.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:51.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:51.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:51.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:51.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.863+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:51.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:51.867+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:53.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.067+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:53.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:53.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:53.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:53.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:53.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.107+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:53.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.107+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:53.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:53.107+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:53.672 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:18:53.676 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:18:53.688 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:18:53.697 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:18:53.783 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:53.783+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:53.784 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:53.783+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:54.773 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:54.775+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:54.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:54.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:54.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:54.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:54.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:54.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.847+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:54.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.847+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:54.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:54.851+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:54.962 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1180091150 2023-10-14T00:18:54.965 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:18:54.984 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242734 2023-10-14T00:18:54.985 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:18:54.985 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1180091150 ]] 2023-10-14T00:18:54.985 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1180091150 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\1\8\0\0\9\1\1\5\0 ]] 2023-10-14T00:18:54.985 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:18:55.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:55.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:55.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:55.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:55.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:55.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:55.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:55.567 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:55.571+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:55.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:55.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:55.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:55.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:56.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:56.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:56.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:56.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:56.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:56.751+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:57.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:57.215+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:57.553 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:57.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:57.568 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:57.571+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:58.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:58.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:58.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:58.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:58.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.115+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:58.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.115+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:58.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.115+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:58.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:58.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:58.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:58.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:58.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.291+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:58.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.291+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:58.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:58.315+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:58.614 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:58.615+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:58.617 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:58.619+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:59.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:59.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:59.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:59.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:59.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:59.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.291+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:59.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.291+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:59.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.311+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:18:59.614 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:59.619+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:59.634 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:59.639+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:59.636 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:59.639+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:59.755 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:59.759+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:18:59.761 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:59.763+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:18:59.775 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:18:59.779+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:18:59.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.835+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:18:59.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:18:59.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:18:59.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:18:59.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:18:59.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:18:59.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.875+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:18:59.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.875+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:18:59.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:18:59.899+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:00.025 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.027+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.027+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb1e1a200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.035+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.035+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb32e5300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.035+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.032 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.035+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325aa00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.038 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e4c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.039 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.039 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d29800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb26bab00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3212900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.043+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2e71580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.067+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.065 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.067+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.067+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb25d5200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.067+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3179000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.071+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.066 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.071+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.071+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.071+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.079+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.079+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d92c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.079+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.079+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.083+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.083+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.083+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2fbf880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.083+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332e780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.087+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.085 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.087+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332e580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.097 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.103+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.098 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.103+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332e380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.107+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.102 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.107+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332ed80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.119+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:19:00.119+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332ec00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:19:01.367 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:01.371+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:01.382 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:01.387+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:03.339 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:03.343+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:03.342 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:03.347+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:03.392 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:03.395+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:05.001 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:19:05.001 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:19:05.008 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:19:05.011 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:19:05.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:05.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:05.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:05.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:05.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:05.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:05.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:05.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:05.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:05.867+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:05.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:05.867+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:05.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:05.891+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:06.657 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1180091150 2023-10-14T00:19:06.658 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:19:06.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242746 2023-10-14T00:19:06.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:19:06.659 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1180091150 ]] 2023-10-14T00:19:06.659 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1180091150 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\1\8\0\0\9\1\1\5\0 ]] 2023-10-14T00:19:06.659 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:19:08.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.227+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:08.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:08.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:08.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:08.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:08.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:08.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.275+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:08.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.275+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:08.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:08.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:08.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:08.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:08.939+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:09.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:09.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:09.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:09.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:09.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:09.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:09.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:09.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:09.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:09.815+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:09.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:09.815+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:10.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:10.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:10.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:10.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:10.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:10.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:10.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.227+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:10.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.359+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:10.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:10.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:10.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:10.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:10.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.407+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:10.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.407+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:10.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:10.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:10.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:10.875+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:10.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:10.875+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:11.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:11.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:11.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:11.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:11.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:11.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:11.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:11.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:13.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:13.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:13.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:13.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:13.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:13.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:13.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:13.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:13.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:13.923+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:13.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:13.923+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:13.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:13.931+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:14.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:14.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:14.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:14.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:14.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:14.363+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:15.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:15.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:16.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:16.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:16.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:16.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.019+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:16.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.019+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:16.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.207+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:16.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:16.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:16.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:16.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:16.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.243+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:16.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.243+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:16.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.243+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:16.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.303+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:16.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:16.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:16.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:16.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:16.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.379+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:16.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.379+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:16.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:16.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:16.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:16.579+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:16.661 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:19:16.661 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:19:16.665 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:19:16.692 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:19:17.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.063+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:17.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:17.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:17.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:17.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:17.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:17.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.107+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:17.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.107+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:17.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:17.123+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:18.168 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1180091150 2023-10-14T00:19:18.168 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:19:18.175 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242758 2023-10-14T00:19:18.175 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:19:18.175 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1180091150 ]] 2023-10-14T00:19:18.175 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1180091150 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\1\8\0\0\9\1\1\5\0 ]] 2023-10-14T00:19:18.176 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:19:19.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.143+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:19.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:19.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:19.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:19.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:19.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.183+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:19.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.183+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:19.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.183+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:19.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:19.395+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:19.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:19.875+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:19.880 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:19.883+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:20.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.455+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:20.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:20.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:20.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:20.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:20.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:20.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.483+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:20.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.483+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:20.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:20.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:20.575 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:20.579+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:20.589 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:20.595+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:20.590 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:20.595+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:20.655 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:20.659+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:20.665 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:20.667+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:20.666 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:20.671+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:22.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:22.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:22.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:22.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:22.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.171+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:22.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.171+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:22.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.179+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:22.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.719+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:22.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:22.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:22.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:22.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:22.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:22.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.815+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:22.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.815+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:22.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:22.831+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:23.687 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:23.691+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:23.693 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:23.695+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:24.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.019+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:24.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:24.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:24.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:24.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:24.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:24.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:24.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:24.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:24.514 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:24.519+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:24.519 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:24.519+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:24.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:24.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:24.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:24.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:24.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:24.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.643+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:24.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.643+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:24.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:24.655+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:25.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:25.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:25.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:25.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:25.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:25.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:25.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:25.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:25.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:25.359+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:25.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:25.375+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:28.187 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:19:28.188 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:19:28.207 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:19:28.225 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:19:28.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.654+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:19:28.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.654+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:19:28.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.666+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:28.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:28.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:28.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:28.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:28.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:28.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.670+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:28.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.670+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:28.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:28.694+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:29.417 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/1180091150 2023-10-14T00:19:29.417 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:19:29.418 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242769 2023-10-14T00:19:29.418 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:19:29.418 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/1180091150 ]] 2023-10-14T00:19:29.419 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/1180091150 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\1\8\0\0\9\1\1\5\0 ]] 2023-10-14T00:19:29.419 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:19:30.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:30.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:30.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:30.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:30.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:30.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.542+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:30.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.542+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:30.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:30.550+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:31.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:31.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:31.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:31.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:31.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:31.278+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:33.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:33.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:33.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:33.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:33.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:33.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:33.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:33.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:33.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:33.906+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:33.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:33.906+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:33.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:33.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:34.463 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:34.466+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:34.479 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:34.482+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:34.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:34.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:34.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.522+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:34.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:34.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:34.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.530+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:34.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.530+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:34.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:34.534+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:35.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:35.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:35.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:35.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:35.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:35.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.126+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:35.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.126+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:35.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:35.130+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:36.307 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:36.310+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:36.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:36.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:37.041 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:37.042+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:37.050 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:37.054+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:37.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:37.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:37.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:37.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:37.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.530+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:37.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.530+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:37.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.566+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:37.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.726+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:37.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:37.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:37.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:37.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:37.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.758+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:37.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.758+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:37.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:37.758+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:37.761 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:37.758+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:19:37.810 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:37.814+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:37.810 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:37.814+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:37.928 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:37.930+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:37.953 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:37.958+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:39.423 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:19:39.423 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:19:39.433 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:19:39.458 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:19:39.568 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:39.570+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:39.602 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:39.606+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:39.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:39.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:39.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:39.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:39.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:39.858+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:40.726 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/323219900 2023-10-14T00:19:40.726 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:19:40.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242780 2023-10-14T00:19:40.732 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:19:40.732 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/323219900 ]] 2023-10-14T00:19:40.732 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/323219900 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\1\8\0\0\9\1\1\5\0 ]] 2023-10-14T00:19:40.732 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/323219900 2023-10-14T00:19:41.225 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:41.230+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:41.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:41.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:41.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:41.262+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:42.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:42.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:42.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:42.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.354+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:42.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:42.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:42.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:42.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.646+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:42.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:42.646+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:43.101 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/323219900 until 2023-10-14T01:19:42.417345+0000 (3600 sec) 2023-10-14T00:19:43.152 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/323219900 2023-10-14T00:19:43.158 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:19:43.454 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:43.458+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:43.455 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:43.458+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:44.391 INFO:tasks.workunit.client.0.smithi183.stderr:listed 18 entries 2023-10-14T00:19:44.430 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/323219900 2023-10-14T00:19:44.430 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:19:44.878 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:19:44.882+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:19:44.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:44.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:44.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:44.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:44.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:44.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:45.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:45.022+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:46.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.046+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:46.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:46.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:46.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:46.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:46.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:46.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.094+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:46.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.094+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:46.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.098+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:46.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.882+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:19:46.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:46.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:46.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:46.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:46.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.918+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:46.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.918+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:46.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:46.926+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:47.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.426+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:47.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:47.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:47.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:47.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:47.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:47.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.458+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:47.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.458+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:47.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.466+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:47.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.550+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:47.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:47.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:47.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:47.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:47.610+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:49.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:49.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:49.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:49.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:49.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:49.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.034+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:49.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.034+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:49.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.038+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:49.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:49.114+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:50.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:50.734+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:50.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:50.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:50.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:50.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:50.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:50.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:50.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:50.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:50.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:50.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:50.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:50.742+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:52.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:52.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:52.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:52.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:52.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.829+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:52.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.829+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:52.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.865+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:52.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.917+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:19:52.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.921+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:19:52.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.925+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:19:52.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:52.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:52.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.941+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:52.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:52.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:53.287 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:53.289+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:53.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:53.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:53.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:53.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:53.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:53.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:53.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:53.849+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:54.048 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:19:54.049+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:19:54.264 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:54.265+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:19:54.437 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:19:54.437 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:19:54.437 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:19:54.454 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:19:56.003 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/323219900 2023-10-14T00:19:56.005 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:19:56.006 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242796 2023-10-14T00:19:56.006 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:19:56.007 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/323219900 ]] 2023-10-14T00:19:56.007 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/323219900 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\3\2\1\9\9\0\0 ]] 2023-10-14T00:19:56.007 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:19:56.519 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:56.521+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:19:56.572 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:19:56.573+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:19:57.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:57.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:57.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:57.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:57.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:57.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:57.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:19:57.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:57.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:19:57.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:19:57.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.697+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:19:57.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.697+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:19:57.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:57.713+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:19:57.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:58.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:19:58.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:58.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:19:58.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:19:58.049+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:00.022 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.025+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.025+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.034 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3492c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.036 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.036 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.037+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253c500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.045+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.045+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332ed80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332eb00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332e980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.085 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.085 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.085+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.093+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.093+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332e180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.093+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.093+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.097+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.097+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.097 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.097+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.097 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.097+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332fa00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.133+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.133+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332fc00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.138 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.141+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.139 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.141+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332fe00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.139 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.141+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.140 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.141+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.140 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.141+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.140 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.141+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.145+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:20:00.145+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:20:01.211 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:01.209+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:01.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:01.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:01.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:01.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:01.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:01.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:01.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:01.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:01.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:01.697 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:01.697+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:02.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:02.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:02.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:02.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:02.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.057+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:02.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.057+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:02.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.065+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:02.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.265+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:02.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:02.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:02.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:02.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:02.313+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:03.157 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:03.157+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:20:03.177 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:03.181+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:20:03.197 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:03.197+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:03.209 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:03.209+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:04.467 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:04.469+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:04.684 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:04.685+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:04.703 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:04.705+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:06.010 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:20:06.029 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:20:06.031 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:20:06.031 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:20:07.690 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/323219900 2023-10-14T00:20:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:20:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242807 2023-10-14T00:20:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:20:07.692 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/323219900 ]] 2023-10-14T00:20:07.692 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/323219900 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\3\2\1\9\9\0\0 ]] 2023-10-14T00:20:07.692 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:20:08.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:08.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:08.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:08.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:08.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.549+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:08.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.549+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:08.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.557+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:08.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.689+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:08.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:08.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:08.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:08.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:08.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:08.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.777+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:08.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.777+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:08.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.777+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:08.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:08.989+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:09.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:09.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:09.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:09.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.045+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:09.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.425+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:20:09.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:09.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:09.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:09.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:09.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:09.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:09.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:09.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:09.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:09.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:12.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.445+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:12.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:12.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:12.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:12.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.489+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:12.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.489+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:12.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.489+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:12.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:12.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:12.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:12.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:12.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.721+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:12.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.721+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:12.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.721+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:12.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:12.775 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:12.777+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:12.794 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:12.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:12.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.961+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:12.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:12.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:12.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:12.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:12.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:12.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:12.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:12.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:13.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:13.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:13.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:13.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:13.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:13.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:13.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:13.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:13.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:13.317+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:13.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:13.325+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:14.115 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:14.117+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:14.115 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:14.117+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:14.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.577+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:14.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:14.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:14.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:14.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:14.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:14.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.645+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:14.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.657+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:14.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:14.669+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:15.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.941+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:15.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:15.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:15.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:15.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:15.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:15.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.961+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:15.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.961+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:15.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:15.993+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:16.034 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:16.037+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:16.053 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:16.053+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:16.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:16.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:16.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:16.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:16.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.333+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:16.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:16.341+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:17.701 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:20:17.702 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:20:17.704 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:20:17.722 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:20:19.226 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/323219900 2023-10-14T00:20:19.234 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:20:19.234 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242819 2023-10-14T00:20:19.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:20:19.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/323219900 ]] 2023-10-14T00:20:19.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/323219900 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\3\2\1\9\9\0\0 ]] 2023-10-14T00:20:19.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:20:19.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:19.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:19.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:19.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:19.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:19.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:19.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:19.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:19.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:19.868+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:19.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:19.868+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:19.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:19.884+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:25.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:25.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:25.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:25.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:25.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:25.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:25.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:25.612+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:25.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:25.612+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:25.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:25.612+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:28.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:28.512+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:20:28.547 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:28.548+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:29.239 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:20:29.242 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:20:29.242 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:20:29.242 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:20:29.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:29.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:29.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:29.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:29.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:29.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.412+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:29.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.412+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:29.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.412+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:29.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.668+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:20:29.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.668+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:20:29.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.668+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:29.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:29.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:29.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:29.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:29.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:29.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.684+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:29.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.684+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:29.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:29.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:29.793 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:29.792+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:30.143 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:30.144+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:30.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:30.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:30.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:30.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:30.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:30.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.392+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:30.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.392+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:30.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.408+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:30.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:30.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:30.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:30.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:30.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.628+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:30.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.628+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:30.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:30.628+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:30.980 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/323219900 2023-10-14T00:20:30.985 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:20:30.986 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242830 2023-10-14T00:20:30.986 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:20:30.987 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/323219900 ]] 2023-10-14T00:20:30.987 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/323219900 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\3\2\1\9\9\0\0 ]] 2023-10-14T00:20:30.987 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:20:31.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:31.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:31.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:31.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:31.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:31.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:31.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:31.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:31.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:31.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:31.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:31.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:31.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:31.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:32.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.284+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:32.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:32.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:32.322 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:32.324+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:32.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:32.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:32.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:32.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.340+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:32.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:32.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.716+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:32.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:32.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:32.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:32.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:32.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:32.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.752+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:32.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.752+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:32.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:32.752+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:32.973 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:32.972+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:20:32.978 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:32.980+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:32.993 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:32.992+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:34.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:34.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:34.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:34.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:34.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:34.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.672+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:34.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:34.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:34.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:34.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:34.724+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:35.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:35.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:35.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:35.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:35.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:35.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.704+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:35.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.704+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:35.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:35.716+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:36.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:36.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:36.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:36.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:36.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.488+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:36.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.488+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:36.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.488+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:36.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.560+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:36.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:36.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:36.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:36.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:36.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:36.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.612+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:36.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.612+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:36.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:36.612+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:37.058 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:37.056+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:37.081 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:37.080+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:38.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:38.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:38.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:38.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:38.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:38.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.024+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:38.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.036+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:38.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.364+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:38.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:38.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:38.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:38.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:38.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:38.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:38.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:38.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.456+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:38.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:38.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:41.006 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:20:41.006 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:20:41.016 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:20:41.022 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:20:41.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.176+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:41.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:41.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:41.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:41.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:41.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:41.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.192+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:41.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.192+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:41.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:41.192+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:42.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:42.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:42.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:42.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:42.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:42.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:42.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:42.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:42.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:42.236+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:42.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:42.256+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:42.392 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/323219900 2023-10-14T00:20:42.393 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:20:42.393 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242842 2023-10-14T00:20:42.393 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:20:42.394 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/323219900 ]] 2023-10-14T00:20:42.394 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/323219900 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\3\2\1\9\9\0\0 ]] 2023-10-14T00:20:42.394 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:20:43.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:43.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:43.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:43.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:43.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:43.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:43.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:43.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:43.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:43.440+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:43.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:43.440+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:45.263 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:45.268+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:45.292 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:45.296+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:47.243 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:47.247+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:47.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.335+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:47.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:47.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:47.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:47.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:47.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.395+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:47.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.399+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:47.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.403+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:47.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:47.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:47.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:47.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:47.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:47.767+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:48.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.275+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:48.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:48.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:48.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:48.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:48.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:48.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.299+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:48.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:48.299+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:49.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:49.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:49.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:49.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:49.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:49.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.247+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:49.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.247+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:49.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:49.267+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:50.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:50.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:50.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:50.159+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:50.493 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:50.495+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:20:50.495 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:50.499+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:20:50.504 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:50.503+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:50.508 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:50.511+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:51.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:51.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:51.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:51.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:52.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:52.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:52.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:52.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:52.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:52.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:52.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:52.011+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:52.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:52.011+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:52.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:52.015+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:52.396 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:20:52.399 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:20:52.401 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:20:52.413 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:20:53.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.207+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:53.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:53.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:53.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:53.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.311+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:53.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:53.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.319+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:53.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.319+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:53.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.327+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:53.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.519+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:20:53.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.539+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:53.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:53.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:53.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:53.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:53.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:53.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.631+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:53.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.631+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:53.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:53.635+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:53.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/193461531 2023-10-14T00:20:53.712 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:20:53.713 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242853 2023-10-14T00:20:53.713 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:20:53.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/193461531 ]] 2023-10-14T00:20:53.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/193461531 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\3\2\1\9\9\0\0 ]] 2023-10-14T00:20:53.714 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/193461531 2023-10-14T00:20:54.675 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:54.679+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:20:54.687 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:54.691+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:20:54.700 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:20:54.703+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:20:56.165 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/193461531 until 2023-10-14T01:20:55.514449+0000 (3600 sec) 2023-10-14T00:20:56.243 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:20:56.252 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/193461531 2023-10-14T00:20:56.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.903+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:56.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:56.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.943+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:56.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:56.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:56.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:56.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.947+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:56.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.947+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:56.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:56.947+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:57.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:57.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:57.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:57.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:57.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.179+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:57.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.179+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:57.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.179+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:57.214 INFO:tasks.workunit.client.0.smithi183.stderr:listed 19 entries 2023-10-14T00:20:57.270 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/193461531 2023-10-14T00:20:57.270 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:20:57.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:57.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:57.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:57.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.807+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:57.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.807+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:57.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:57.819+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:58.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.575+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:20:58.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.587+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:20:58.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:58.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:20:58.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:58.602 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:20:58.607+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:20:58.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:58.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:58.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.663+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:58.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.663+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:58.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.663+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:20:58.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:20:58.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:20:58.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:20:58.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:20:58.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.907+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:20:58.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.907+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:20:58.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:20:58.919+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:00.007 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.011+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.008 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.011+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.008 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.011+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.009 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.011+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3492800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.009 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.011+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.009 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.011+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.015+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.015+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.021 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.023+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.021 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.023+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e4b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.031+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.031+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.057 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.059+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.057 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.059+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.063+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.063+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256a80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.063+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.063+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.063+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.063+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.068 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.071+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.069 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.071+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.087+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.084 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.087+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325ab00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.115+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.115+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.119+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.119+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.121 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.123+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.122 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.123+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.128 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.127+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.129 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.127+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257a00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.147+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.146 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.147+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.146 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.147+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.146 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.147+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.152 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.155+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.152 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.155+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.152 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.155+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c50000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.152 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:21:00.155+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c50200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:21:00.494 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:00.499+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:00.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:00.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:00.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:00.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:00.511 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:00.515+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:00.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:00.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:00.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:00.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:00.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:00.531+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:00.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:00.531+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:00.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:00.547+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:01.037 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:01.039+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:01.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:01.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:01.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:01.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:01.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:01.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.151+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:01.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.151+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:01.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.151+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:01.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.835+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:01.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:01.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:01.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:01.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:01.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.855+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:01.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.855+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:01.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:01.855+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:02.278 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:02.283+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:02.286 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:02.287+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:03.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.115+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:03.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:03.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:03.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:03.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:03.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:03.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.163+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:03.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.163+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:03.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:03.171+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:05.600 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:05.603+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:06.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:06.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:06.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:06.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:06.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:06.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.111+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:06.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.111+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:06.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.127+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:06.319 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:06.323+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:06.356 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:06.355+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:06.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:06.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:06.823+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:07.272 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:21:07.272 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:21:07.273 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:21:07.301 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:21:07.461 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:07.463+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:07.466 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:07.471+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:07.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:07.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:07.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:07.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:07.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:07.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:07.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:07.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:07.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:07.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:07.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:07.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:08.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:08.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:08.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:08.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:08.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:08.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.415+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:08.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.415+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:08.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.459+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:08.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:08.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:08.921 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/193461531 2023-10-14T00:21:08.926 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:21:08.927 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242868 2023-10-14T00:21:08.927 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:21:08.928 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/193461531 ]] 2023-10-14T00:21:08.928 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/193461531 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\9\3\4\6\1\5\3\1 ]] 2023-10-14T00:21:08.928 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:21:09.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:09.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:09.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:09.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:09.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:09.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.651+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:09.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.651+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:09.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:09.683+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:09.804 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:09.807+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:10.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:10.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:10.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:10.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:10.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:10.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:10.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.719+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:10.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:10.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:10.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:11.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:11.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:11.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:11.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.095+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:11.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.095+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:11.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.103+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:11.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.103+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:21:11.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.103+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:21:11.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:11.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:11.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:11.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:11.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.651+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:11.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:11.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:11.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:11.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:11.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:11.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.707+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:11.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.707+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:11.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:11.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:13.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:13.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:13.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:13.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:13.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:13.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:13.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:13.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:13.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:13.739+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:13.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:13.739+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:13.751+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:14.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:14.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:14.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:14.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:14.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.791+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:14.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.791+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:14.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.791+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:14.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.815+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:14.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:14.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:14.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:15.847 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:15.851+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:15.867 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:15.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:16.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.019+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:16.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:16.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:16.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:16.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:16.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:16.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.055+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:16.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.055+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:16.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.055+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:16.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:16.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:16.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:16.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.235+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:16.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.235+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:16.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.259+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:16.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.967+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:16.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:16.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:16.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:16.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:16.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:16.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.987+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:16.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:16.987+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:17.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:17.003+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:18.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:18.198+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:18.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:18.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:18.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:18.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:18.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:18.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:18.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:18.218+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:18.930 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:21:18.931 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:21:18.934 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:21:18.947 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:21:20.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.142+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:20.504 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/193461531 2023-10-14T00:21:20.504 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:21:20.505 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242880 2023-10-14T00:21:20.505 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:21:20.505 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/193461531 ]] 2023-10-14T00:21:20.505 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/193461531 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\9\3\4\6\1\5\3\1 ]] 2023-10-14T00:21:20.506 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:21:20.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:20.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:20.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:20.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:20.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.890+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:20.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.890+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:20.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:20.898+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:21.006 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:21.006+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:21:21.007 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:21.010+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:21.028 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:21.030+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:21.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.650+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:21.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:21.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:21.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:21.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:21.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:21.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:21.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:21.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.678+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:21.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.986+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:21:21.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.986+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:21:21.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.986+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:21.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:21.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:21.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:21.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:21.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:21.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:21.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:21.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:23.624 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:23.626+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:21:23.646 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:23.650+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:23.646 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:23.650+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:24.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.890+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:24.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:24.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:24.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:24.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:24.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:24.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:24.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:24.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:26.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.126+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:26.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:26.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:26.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:26.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:26.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:26.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.186+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:26.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.186+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:26.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:26.206+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:27.440 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:27.434+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:28.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.798+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:28.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:28.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:28.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:28.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:28.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.834+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:28.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.834+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:28.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:28.834+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:29.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.050+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:29.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:29.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:29.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:29.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:29.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.066+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:29.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.066+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:29.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:29.066+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:29.837 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:29.838+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:30.511 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:21:30.511 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:21:30.514 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:21:30.514 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:21:30.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:30.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:30.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:30.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:30.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:30.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:30.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:30.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:30.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:30.774+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:30.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:30.774+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:30.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:30.786+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:31.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:31.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:31.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:31.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:31.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.014+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:31.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.042+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:31.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:31.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.330+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:31.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:31.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:31.872 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/193461531 2023-10-14T00:21:31.873 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:21:31.873 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242891 2023-10-14T00:21:31.873 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:21:31.874 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/193461531 ]] 2023-10-14T00:21:31.874 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/193461531 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\9\3\4\6\1\5\3\1 ]] 2023-10-14T00:21:31.874 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:21:32.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:32.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:32.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:32.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:32.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:32.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:32.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:32.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:33.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:33.010+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:33.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:33.010+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:33.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:33.010+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:33.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:33.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:33.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:33.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:33.368 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:33.370+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:34.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:34.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:34.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:34.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:34.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:34.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:34.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:34.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:34.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:34.630+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:34.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:34.630+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:34.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:34.630+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:35.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.282+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:35.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:35.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:35.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:35.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:35.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.318+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:35.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.318+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:35.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:35.322+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:36.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:36.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:36.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:36.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:36.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:36.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:36.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:36.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:36.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:36.326+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:36.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:36.326+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:36.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:36.342+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:37.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:37.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:37.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:37.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:37.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:37.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.726+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:37.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.726+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:37.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:37.738+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:38.009 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:38.006+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:21:38.010 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:38.006+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:38.293 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:38.294+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:21:38.330 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:38.326+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:38.330 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:38.326+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:39.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:39.362+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:39.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:39.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:39.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:39.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:39.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:39.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:39.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:39.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:39.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:39.394+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:39.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:39.406+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:39.849 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:39.850+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:21:39.887 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:39.890+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:41.876 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:21:41.878 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:21:41.881 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:21:41.886 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:21:41.969 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:41.970+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:41.985 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:41.986+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:41.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:41.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:41.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:41.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:41.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:41.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:41.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:41.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:42.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.034+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:42.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.454+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:21:42.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.454+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:21:42.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.454+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:42.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:42.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:42.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:42.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:42.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:42.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.506+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:42.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.506+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:42.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.510+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:42.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:42.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:43.477 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/193461531 2023-10-14T00:21:43.487 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:21:43.488 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242903 2023-10-14T00:21:43.488 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:21:43.488 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/193461531 ]] 2023-10-14T00:21:43.488 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/193461531 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\9\3\4\6\1\5\3\1 ]] 2023-10-14T00:21:43.489 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:21:44.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:44.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:44.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:44.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:44.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:44.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:44.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:44.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:44.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:44.162+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:44.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:44.162+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:45.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:44.998+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:45.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:45.010+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:45.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:45.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:45.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:45.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:45.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:45.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:45.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:45.026+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:45.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:45.026+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:46.680 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:46.682+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:21:46.691 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:46.694+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:46.700 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:46.702+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:46.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.826+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:46.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:46.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:46.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:46.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:46.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:46.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.874+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:46.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.874+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:46.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:46.882+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:48.366 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:48.366+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:49.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:49.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:49.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:49.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:49.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:49.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.202+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:49.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.202+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:49.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:49.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:50.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:50.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:50.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:50.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:50.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:50.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:51.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:51.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:51.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:51.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:51.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:51.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:51.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:51.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:51.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:51.277+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:52.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:52.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:52.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:52.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:52.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:52.101+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:52.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:52.101+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:52.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:52.101+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:53.505 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:21:53.506 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:21:53.506 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:21:53.537 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:21:54.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:54.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:54.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:54.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:54.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.413+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:54.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.413+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:54.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.413+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:54.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.485+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:54.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:54.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:54.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:54.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:54.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.541+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:54.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.541+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:54.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:54.541+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:54.886 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2831995567 2023-10-14T00:21:54.886 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:21:54.887 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242914 2023-10-14T00:21:54.887 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:21:54.888 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2831995567 ]] 2023-10-14T00:21:54.888 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2831995567 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\9\3\4\6\1\5\3\1 ]] 2023-10-14T00:21:54.888 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2831995567 2023-10-14T00:21:56.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.045+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:21:56.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:56.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:56.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:56.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:56.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.069+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:56.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.069+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:56.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:56.069+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:57.188 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2831995567 until 2023-10-14T01:21:56.503006+0000 (3600 sec) 2023-10-14T00:21:57.231 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2831995567 2023-10-14T00:21:57.231 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:21:57.265 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:57.265+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:21:57.282 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:57.281+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:57.535 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:57.533+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:21:57.539 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:21:57.541+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:21:57.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:57.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:57.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:57.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:57.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:57.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.649+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:57.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.649+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:57.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:57.657+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:21:58.396 INFO:tasks.workunit.client.0.smithi183.stderr:listed 20 entries 2023-10-14T00:21:58.498 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2831995567 2023-10-14T00:21:58.499 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:21:59.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:21:59.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:21:59.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:21:59.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:21:59.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:21:59.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.221+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:21:59.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.221+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:21:59.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:21:59.233+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:00.000 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.001+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.000 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.001+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb253d180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.001+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.001+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb325ae80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.009 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.009+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.009+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2fbf280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.012 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.013+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.013+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2d92b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.017+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.016 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.017+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3212580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.022 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.025+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.025+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb32e4e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.026 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.029+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.029+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb332e580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.029+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.027 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.029+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb32e5400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.049 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.049+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.049 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.049+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c50380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2e70380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332ea00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c50980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.073+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c50b80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.077 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.077+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c50580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.113 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.113+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c50d80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.117+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.121+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c50f80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.121+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.121+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.121+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.121+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c51180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.121+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c51380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.121+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c51d80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.125+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:22:00.125+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c51c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:22:00.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.333+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:00.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.349+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:22:00.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.349+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:00.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:00.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:00.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:00.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:00.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:00.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.377+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:00.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.377+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:00.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:00.385+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:01.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:01.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:01.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:01.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:01.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:01.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:01.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:01.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:01.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:01.441+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:01.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:01.441+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:01.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:01.441+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:02.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.877+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:02.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.877+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:22:02.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.877+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:02.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:02.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:02.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:02.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:02.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:02.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.893+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:02.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.893+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:02.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:02.893+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:03.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:03.597+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:03.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:03.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:03.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:03.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:03.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:03.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:03.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:03.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:03.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:03.645+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:03.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:03.645+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:03.791 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:03.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:22:04.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:04.341+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:06.163 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:06.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:22:06.203 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:06.205+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:07.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:07.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:07.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:07.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:07.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.313+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:07.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.313+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:07.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.313+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:07.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:07.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:07.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:08.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.005+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:08.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.005+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:22:08.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.009+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:08.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:08.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:08.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:08.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:08.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:08.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.017+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:08.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.017+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:08.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.017+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:08.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.317+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:08.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.321+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:08.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:08.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:08.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:08.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:08.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:08.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.429+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:08.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.429+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:08.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:08.429+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:08.515 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:22:08.537 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:22:08.537 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:22:08.543 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:22:09.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.157+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:09.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:09.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:09.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:09.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:09.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:09.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.165+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:09.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.201+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:09.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:09.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:09.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:09.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:09.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.829+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:09.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.829+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:09.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:09.829+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:09.830 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2831995567 2023-10-14T00:22:09.831 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:22:09.832 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242929 2023-10-14T00:22:09.832 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:22:09.832 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2831995567 ]] 2023-10-14T00:22:09.833 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2831995567 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\8\3\1\9\9\5\5\6\7 ]] 2023-10-14T00:22:09.833 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:22:10.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:10.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:10.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:10.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:10.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:10.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:10.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:10.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:10.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:11.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:11.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:11.001+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:11.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:11.001+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:11.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:11.001+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:12.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.065+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:12.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:12.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:12.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:12.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:12.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:12.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.097+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:12.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.097+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:12.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.097+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:12.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.701+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:12.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:12.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:12.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:12.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:12.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:12.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.765+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:12.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.765+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:12.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:12.781+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:13.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:13.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:13.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:13.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:13.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:13.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:13.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:13.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:13.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:13.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:13.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:13.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:13.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:13.401+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:15.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.189+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:15.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:15.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:15.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:15.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:15.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:15.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.213+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:15.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:15.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:15.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:15.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:15.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.685+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:15.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.685+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:15.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:15.685+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:15.974 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:15.973+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:16.754 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:16.753+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:17.731 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:17.733+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:18.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.209+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:18.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:18.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:18.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:18.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:18.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:18.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.245+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:18.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.245+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:18.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:18.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:19.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:19.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:19.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:19.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:19.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.485+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:19.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:19.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:19.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:19.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:19.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:19.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:19.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.797+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:19.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.797+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:19.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:19.817+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:19.839 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:22:19.846 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:22:19.846 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:22:19.852 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:22:20.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:20.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:20.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:20.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:20.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:20.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:20.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:20.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:20.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:21.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:21.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:21.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:21.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:21.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:21.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.061+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:21.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.065+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:21.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:21.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:21.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:21.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:21.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:21.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.225+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:21.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.225+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:21.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.245+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:21.279 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2831995567 2023-10-14T00:22:21.280 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:22:21.280 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242941 2023-10-14T00:22:21.281 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:22:21.281 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2831995567 ]] 2023-10-14T00:22:21.281 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2831995567 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\8\3\1\9\9\5\5\6\7 ]] 2023-10-14T00:22:21.281 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:22:21.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:21.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:21.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:21.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:21.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.377+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:21.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:21.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:21.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:21.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:21.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.757+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:21.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.757+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:21.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:21.757+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:22.137 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:22.137+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:22.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:22.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:22.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:22.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:22.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:22.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:22.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:22.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:22.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:22.733+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:22.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:22.733+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:22.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:22.741+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:23.313 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:23.313+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:22:23.343 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:23.345+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:24.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:23.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:24.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:24.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:24.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:24.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:24.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.021+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:24.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.021+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:24.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.021+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:24.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:24.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:24.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:24.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.969+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:24.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.969+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:24.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:24.973+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:25.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.101+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:25.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:25.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:25.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:25.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:25.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.125+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:25.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.125+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:25.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.129+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:25.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.536+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:25.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:25.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:25.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:25.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:25.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:25.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.596+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:25.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.596+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:25.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.632+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:25.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.632+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:22:25.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:25.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:25.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:25.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:25.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:25.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.688+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:25.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:25.688+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:26.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:26.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:26.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:26.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:26.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:26.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:26.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:26.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:26.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:26.112+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:26.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:26.112+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:26.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:26.156+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:27.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.392+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:27.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:27.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:27.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:27.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:27.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:27.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.436+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:27.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.436+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:27.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.436+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:27.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:27.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:28.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:28.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:28.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:28.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:28.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:28.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.492+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:28.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.492+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:28.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:28.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:28.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:28.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:29.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:29.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:29.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:29.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:29.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:29.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:29.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:29.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:29.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:29.204+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:29.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:29.208+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:29.666 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:29.664+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:22:29.693 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:29.692+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:22:29.694 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:29.692+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:30.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:30.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:30.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:30.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:30.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:30.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.456+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:30.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.456+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:30.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.460+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:30.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:30.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:30.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:30.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:30.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:30.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:30.916+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:31.282 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:22:31.293 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:22:31.311 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:22:31.316 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:22:31.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:31.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:31.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:31.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:31.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:31.816+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:32.665 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2831995567 2023-10-14T00:22:32.666 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:22:32.697 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242952 2023-10-14T00:22:32.697 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:22:32.698 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2831995567 ]] 2023-10-14T00:22:32.698 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2831995567 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\8\3\1\9\9\5\5\6\7 ]] 2023-10-14T00:22:32.699 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:22:33.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:33.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:33.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:33.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:33.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:33.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:33.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:33.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:33.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:33.704+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:33.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:33.704+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:33.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:33.720+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:34.198 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:34.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:22:35.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.904+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:35.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:35.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:35.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:35.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:35.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.960+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:35.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.960+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:35.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:35.960+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:37.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:37.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:37.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:37.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:37.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:37.008+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:39.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:39.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:39.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:39.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:39.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:39.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.876+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:39.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.904+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:39.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:39.904+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:40.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.064+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:22:40.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.072+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:40.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:40.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:40.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:40.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:40.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:40.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.120+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:40.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.120+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:40.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:40.120+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:42.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.572+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:42.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:42.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:42.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.616+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:42.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:42.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:42.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.632+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:42.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.632+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:42.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:42.636+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:42.713 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:22:42.722 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:22:42.724 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:22:42.726 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:22:43.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:43.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:43.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:43.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.192+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:43.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.192+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:43.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.200+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:43.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.484+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:43.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:43.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:43.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:43.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:43.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.524+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:43.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.524+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:43.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:43.524+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:44.042 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2831995567 2023-10-14T00:22:44.043 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:22:44.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242964 2023-10-14T00:22:44.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:22:44.045 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2831995567 ]] 2023-10-14T00:22:44.045 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2831995567 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\8\3\1\9\9\5\5\6\7 ]] 2023-10-14T00:22:44.045 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:22:46.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:46.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:46.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:46.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:46.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.128+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:46.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.128+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:46.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.132+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:46.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:46.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:46.916+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:46.917 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:46.916+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:22:46.943 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:46.944+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:22:47.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.356+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:47.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:47.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:47.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:47.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:47.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.392+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:47.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.392+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:47.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.392+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:47.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.728+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:47.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:47.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:47.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:47.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:47.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:47.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.804+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:47.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.804+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:47.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:47.816+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:49.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.508+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:49.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.508+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:22:49.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:49.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:49.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:49.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:49.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:49.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.588+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:49.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.588+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:49.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.588+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:49.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.832+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:49.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:49.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:49.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:49.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:49.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:49.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:49.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:49.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.908+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:49.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.920+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:22:49.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.920+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:49.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:49.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:49.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:49.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:49.940+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:50.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.292+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:50.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:50.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:50.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:50.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:50.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.680+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:50.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.680+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:50.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:50.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:51.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:51.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:51.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:51.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:51.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:51.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.128+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:51.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.128+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:51.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.148+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:51.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.188+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:51.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:51.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:51.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:51.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:51.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:51.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.208+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:51.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.208+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:51.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.212+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:51.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:51.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:51.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:53.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:53.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:53.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:53.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:53.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:53.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:53.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:53.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:53.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:53.784+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:53.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:53.784+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:53.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:53.792+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:54.046 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:22:54.047 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:22:54.047 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:22:54.049 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:22:55.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.196+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:55.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:55.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:55.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:55.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:55.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:55.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.256+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:55.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.256+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:55.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.256+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:55.301 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2831995567 2023-10-14T00:22:55.301 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:22:55.319 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242975 2023-10-14T00:22:55.319 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:22:55.319 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2831995567 ]] 2023-10-14T00:22:55.319 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2831995567 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\8\3\1\9\9\5\5\6\7 ]] 2023-10-14T00:22:55.319 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:22:55.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.644+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:55.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:55.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:55.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:55.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:55.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:55.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.680+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:55.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.680+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:55.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:55.684+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:55.835 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:22:55.840+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:22:57.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:57.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:57.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:57.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:57.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:57.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:57.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:57.164+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:58.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:58.136+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:22:58.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:58.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:22:58.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:58.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:58.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:58.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:58.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:58.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:58.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:58.600+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:58.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:58.600+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:59.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.284+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:22:59.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:22:59.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:22:59.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:22:59.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:22:59.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.324+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:22:59.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.324+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:22:59.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:22:59.324+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:00.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:00.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:00.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:00.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.172+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:00.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.172+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:00.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.180+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:00.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:00.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:00.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:00.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:00.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.372+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:00.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.372+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:00.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.372+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:00.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.372+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:00.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:00.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:00.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:00.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:00.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.408+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:00.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:00.408+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:01.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:01.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:01.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:01.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:01.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:01.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:01.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:01.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:01.092+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:01.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:01.096+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:02.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:02.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:02.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:02.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:02.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:02.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:02.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:02.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:02.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:02.160+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:02.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:02.160+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:02.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:02.168+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:04.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:04.647+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:05.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.127+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:05.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:05.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:05.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:05.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:05.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.159+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:05.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.159+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:05.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.159+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:05.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.251+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:05.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:05.272 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:05.271+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:05.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:05.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:05.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:05.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:05.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.279+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:05.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.279+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:05.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:05.279+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:05.320 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:23:05.327 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:23:05.332 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:23:05.340 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:23:06.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:06.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:06.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:06.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.135+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:06.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:06.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.283+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:06.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:06.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:06.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:06.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:06.339+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:06.678 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/557665774 2023-10-14T00:23:06.678 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:23:06.679 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697242986 2023-10-14T00:23:06.679 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:23:06.679 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/557665774 ]] 2023-10-14T00:23:06.679 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/557665774 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\8\3\1\9\9\5\5\6\7 ]] 2023-10-14T00:23:06.680 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/557665774 2023-10-14T00:23:06.770 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:06.771+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:23:06.782 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:06.783+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:23:06.792 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:06.795+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:07.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:07.259+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:07.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:07.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:07.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:07.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:07.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:07.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:07.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:07.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:07.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:07.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:07.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:07.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:08.373 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:08.375+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:09.078 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:09.079+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:09.186 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/557665774 until 2023-10-14T01:23:08.506733+0000 (3600 sec) 2023-10-14T00:23:09.287 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:23:09.292 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/557665774 2023-10-14T00:23:09.585 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:09.587+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:23:09.592 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:09.595+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:23:09.592 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:09.595+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:10.602 INFO:tasks.workunit.client.0.smithi183.stderr:listed 21 entries 2023-10-14T00:23:10.664 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/557665774 2023-10-14T00:23:10.664 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:23:11.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:11.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:11.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:11.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:11.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:11.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:11.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:11.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:11.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:11.303+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:11.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:11.303+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:11.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:11.303+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:13.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:13.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:13.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:13.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:13.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:13.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:13.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:13.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:13.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:13.355+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:16.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:16.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:16.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:16.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:16.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.267+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:16.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.267+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:16.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.275+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:16.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:16.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.615+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:16.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:16.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:16.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:16.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:16.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:16.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:16.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:17.428 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:17.431+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:23:17.456 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:17.459+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:18.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:18.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:18.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:18.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:18.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:18.063+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:19.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.667+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:19.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:19.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:19.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:19.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:19.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:19.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.683+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:19.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.683+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:19.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:19.683+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:20.679 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:23:20.681 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:23:20.701 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:23:20.702 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:23:21.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.639+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:23:21.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.647+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:21.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:21.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:21.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:21.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.695+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:21.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.723+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:23:21.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.723+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:23:21.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.735+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:21.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:21.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:21.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:21.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:21.783+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:22.482 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/557665774 2023-10-14T00:23:22.490 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:23:22.492 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243002 2023-10-14T00:23:22.492 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:23:22.493 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/557665774 ]] 2023-10-14T00:23:22.493 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/557665774 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\5\7\6\6\5\7\7\4 ]] 2023-10-14T00:23:22.493 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:23:23.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:23.983+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:23.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:23.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:24.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:24.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:24.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:24.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:24.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:24.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:24.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:24.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:24.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:24.043+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:24.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:24.043+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:24.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:26.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:26.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:26.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:26.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:26.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:26.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.271+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:26.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.271+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:26.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:26.271+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:30.128 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:30.131+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:23:30.137 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:30.139+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:30.347 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:30.351+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:23:30.391 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:30.395+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:32.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:32.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:32.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:32.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.079+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:32.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.079+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:32.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:32.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:32.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:32.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:32.475+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:32.506 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:23:32.510 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:23:32.526 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:23:32.526 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:23:33.814 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/557665774 2023-10-14T00:23:33.817 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:23:33.818 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243013 2023-10-14T00:23:33.818 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:23:33.818 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/557665774 ]] 2023-10-14T00:23:33.819 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/557665774 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\5\7\6\6\5\7\7\4 ]] 2023-10-14T00:23:33.819 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:23:35.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:35.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:35.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:35.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:35.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:35.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.111+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:35.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.131+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:35.524 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:35.527+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:23:35.538 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:35.535+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:35.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:35.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:35.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:35.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.775+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:35.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:35.775+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:36.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:36.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:36.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:36.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:36.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:36.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:36.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:36.899+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:37.643 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:37.647+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:23:37.653 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:37.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:37.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:37.991+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:23:38.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.011+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:38.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:38.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:38.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:38.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:38.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:38.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.091+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:38.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.091+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:38.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.091+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:38.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:38.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:38.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:38.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:38.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:38.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.727+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:38.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.727+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:38.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:38.763+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:39.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:39.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:39.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:39.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:39.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:39.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:39.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:39.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:39.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:39.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:40.037 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:40.039+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:23:40.046 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:40.051+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:41.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:41.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:41.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:41.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:41.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:41.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:41.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:41.547+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:41.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:41.547+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:42.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:42.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:42.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:42.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:42.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:42.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:42.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:42.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:42.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:42.835+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:42.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:42.835+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:42.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:42.835+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:43.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:43.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:43.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:43.067+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:43.850 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:23:43.850 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:23:43.850 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:23:43.870 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:23:44.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:44.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:44.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:44.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:44.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.150+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:44.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.150+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:44.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:44.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:44.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:44.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:44.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:44.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.998+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:44.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:44.998+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:45.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/557665774 2023-10-14T00:23:45.440 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:23:45.441 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243025 2023-10-14T00:23:45.442 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:23:45.442 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/557665774 ]] 2023-10-14T00:23:45.442 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/557665774 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\5\7\6\6\5\7\7\4 ]] 2023-10-14T00:23:45.442 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:23:45.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:45.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:45.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:45.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:45.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:45.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:46.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:46.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:46.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:46.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:46.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:46.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:46.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:46.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:46.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:46.070+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:46.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:46.070+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:46.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:46.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:46.765 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:46.766+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:49.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.750+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:49.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:49.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:49.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:49.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:49.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:49.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.782+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:49.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.782+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:49.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:49.806+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:50.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.446+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:23:50.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.446+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.446+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:50.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:50.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:50.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:50.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:50.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.590+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:50.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.590+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:50.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.610+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:23:50.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.614+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:23:50.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:50.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:50.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:50.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:50.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:50.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.658+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:50.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.658+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:50.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:50.950+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:54.389 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:54.390+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:23:54.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:54.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:54.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:54.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:54.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:54.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:54.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:54.726+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:54.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:54.742+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:55.444 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:23:55.445 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:23:55.447 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:23:55.476 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:23:56.732 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/557665774 2023-10-14T00:23:56.732 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:23:56.732 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243036 2023-10-14T00:23:56.733 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:23:56.733 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/557665774 ]] 2023-10-14T00:23:56.733 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/557665774 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\5\7\6\6\5\7\7\4 ]] 2023-10-14T00:23:56.733 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:23:57.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.230+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:57.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:57.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:57.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:57.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:57.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:57.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.298+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:57.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.298+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:57.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:57.298+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:58.181 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:23:58.182+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:23:58.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:58.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:58.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:58.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:58.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.266+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:58.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.266+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:58.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.266+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:58.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:58.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:58.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:58.498+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:59.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:59.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:59.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:59.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:59.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.098+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:59.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.098+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:23:59.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.182+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:23:59.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:23:59.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:23:59.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:23:59.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:23:59.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:23:59.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:23:59.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:23:59.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:23:59.218+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:00.003 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.006+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.004 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.006+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb325bd00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.009 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.010+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.009 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.010+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.009 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.010+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.010+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3257080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.014+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.013 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.014+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.030 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.030+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.030 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.030+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3257a80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.046 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.050+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.047 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.050+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.054 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.054+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.054 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.054+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.078 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb332f800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb332f200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb332fe80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.078+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3493c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.086 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.090+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.090+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3492600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.106+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.106+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3258900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.106+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.106+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb253dc80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.114+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.114+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb332e380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.114+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.114+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c51580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.118+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.118+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c51800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.122+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.120 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.122+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c51a00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.121 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.122+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.122 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.122+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3259a00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.133 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.134+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.134 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:24:00.134+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2c51f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:24:00.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:00.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:00.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:00.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:00.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:00.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:00.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:00.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:02.605 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:02.606+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:02.617 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:02.618+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:03.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:03.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:03.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:03.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:03.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:03.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.046+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:03.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.046+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:03.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.086+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:03.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.594+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:03.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:03.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:03.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:03.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:03.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:03.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.630+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:03.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.630+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:03.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:03.630+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:04.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:04.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:04.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:04.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:04.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:04.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:04.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:04.202+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:05.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:05.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:05.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:05.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:05.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.046+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:05.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.046+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:05.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:05.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:05.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:05.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:05.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:05.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.334+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:05.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.334+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:05.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.346+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:05.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:05.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:05.569 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:05.566+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:05.607 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:05.610+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:06.734 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:24:06.735 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:24:06.735 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:24:06.749 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:24:07.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.194+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:07.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:07.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:07.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:07.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:07.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:07.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.258+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:07.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.258+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:07.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:07.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:08.085 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/557665774 2023-10-14T00:24:08.095 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:24:08.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243048 2023-10-14T00:24:08.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:24:08.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/557665774 ]] 2023-10-14T00:24:08.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/557665774 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\5\7\6\6\5\7\7\4 ]] 2023-10-14T00:24:08.097 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:24:09.691 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:09.694+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:24:09.696 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:09.698+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:09.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:09.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:09.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:09.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:09.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:09.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:09.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:09.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:09.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:09.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:09.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:09.726+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:09.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:09.750+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:09.751 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:09.750+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:09.751 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:09.750+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:10.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.098+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:10.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:10.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:10.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:10.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.138+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:10.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.146+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:10.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:10.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:10.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:11.835 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:11.838+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:11.835 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:11.838+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:11.836 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:11.838+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:12.442 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:12.446+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:12.443 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:12.446+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:14.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.554+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:24:14.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.562+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:24:14.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.586+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:14.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:14.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:14.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:14.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:14.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:14.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.618+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:14.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.618+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:14.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:14.622+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:15.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.186+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:15.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:15.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:15.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:15.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:15.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:15.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.286+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:15.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.286+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:15.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.294+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:15.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:15.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:15.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:15.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:15.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.622+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:15.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.622+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:15.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:15.638+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:16.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:16.050+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:17.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:17.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:17.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:17.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:17.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.474+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:17.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.474+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:17.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.474+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:17.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.490+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:24:17.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.490+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:24:17.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:17.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:18.069 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:18.070+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:18.099 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:24:18.099 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:24:18.102 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:24:18.104 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:24:19.112 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:19.114+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:19.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:19.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:19.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:19.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:19.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:19.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.494+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:19.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.494+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:19.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:19.494+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:19.592 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/583246470 2023-10-14T00:24:19.604 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:24:19.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243059 2023-10-14T00:24:19.612 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:24:19.612 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/583246470 ]] 2023-10-14T00:24:19.612 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/583246470 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\5\7\6\6\5\7\7\4 ]] 2023-10-14T00:24:19.612 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/583246470 2023-10-14T00:24:20.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.510+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:24:20.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.510+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:24:20.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.546+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:20.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:20.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:20.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:20.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:20.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:20.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.570+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:20.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.570+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:20.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.578+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:20.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.766+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:20.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:20.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:20.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:20.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:20.818+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:22.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:22.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:22.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:22.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.130+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:22.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.130+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:22.149 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/583246470 until 2023-10-14T01:24:21.420720+0000 (3600 sec) 2023-10-14T00:24:22.222 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/583246470 2023-10-14T00:24:22.224 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:24:22.820 INFO:tasks.workunit.client.0.smithi183.stderr:listed 22 entries 2023-10-14T00:24:22.835 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/583246470 2023-10-14T00:24:22.836 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:24:22.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:22.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:22.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:22.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.998+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:22.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:22.998+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:24.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:24.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:24.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:24.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:24.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:24.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:24.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:24.502+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:25.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:25.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:25.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:25.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:25.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:25.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:25.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:25.242+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:25.341 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:25.342+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:26.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:26.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:26.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:26.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:26.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.170+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:26.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.170+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:26.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.238+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:26.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:26.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:26.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:26.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:26.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.294+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:26.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.294+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:26.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:26.294+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:26.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:26.337+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:24:26.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:26.337+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:26.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:26.337+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:27.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:27.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:27.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:27.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:27.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.265+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:27.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:27.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:27.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:27.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:27.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:27.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:27.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.993+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:27.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.993+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:27.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:27.993+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:28.127 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:28.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:28.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:28.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:28.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:28.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:28.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:28.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.981+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:28.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.981+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:28.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:28.993+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:29.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:29.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:29.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:29.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:29.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:29.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.069+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:29.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.069+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:29.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:29.069+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:30.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:30.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:30.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:30.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:30.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:30.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:30.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:30.065+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:30.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:30.065+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:30.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:30.085+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:30.323 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:30.325+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:31.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.197+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:31.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:31.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:31.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:31.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:31.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.253+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:31.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.261+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:31.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.261+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:24:31.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.261+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:31.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:31.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:31.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:31.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:31.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.293+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:31.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.297+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:31.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:31.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:31.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:31.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:31.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.721+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:31.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.721+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:31.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.737+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:31.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:31.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:31.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:31.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:31.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.937+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:31.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.937+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:31.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:31.953+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:32.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:32.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:32.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:32.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:32.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:32.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:32.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.545+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:32.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.561+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:32.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.573+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:32.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.821+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:32.851 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:24:32.856 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:24:32.856 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:24:32.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:32.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:32.878 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:24:32.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:32.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:32.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:32.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.885+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:32.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.885+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:32.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:32.885+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:34.164 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/583246470 2023-10-14T00:24:34.164 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:24:34.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243074 2023-10-14T00:24:34.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:24:34.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/583246470 ]] 2023-10-14T00:24:34.165 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/583246470 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\8\3\2\4\6\4\7\0 ]] 2023-10-14T00:24:34.166 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:24:35.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.189+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:35.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:35.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:35.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:35.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:35.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:35.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.249+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:35.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.249+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:35.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:35.249+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:36.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:36.281+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:38.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:38.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:38.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:38.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:38.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:38.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:38.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:38.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:38.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:38.257+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:38.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:38.261+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:39.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:39.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:39.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:39.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:39.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:39.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.141+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:39.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.141+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:39.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:39.145+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:39.350 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:39.353+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:39.357 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:39.357+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:39.624 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:39.625+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:40.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:40.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:40.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:40.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:40.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:40.225+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:40.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:40.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:41.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:41.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:41.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:41.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:41.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:41.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:41.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:41.053+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:41.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:41.053+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:41.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:41.061+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:41.293 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:41.293+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:41.320 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:41.321+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:41.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:41.321+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:43.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.097+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:43.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:43.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:43.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:43.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:43.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.173+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:43.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.173+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:43.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:43.173+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:44.176 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:24:44.176 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:24:44.189 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:24:44.192 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:24:44.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:44.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:44.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:44.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:44.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:44.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.685+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:44.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.685+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:44.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:44.721+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:45.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:45.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:45.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:45.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:45.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:45.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:45.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.217+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:45.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:45.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.437+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:45.458 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/583246470 2023-10-14T00:24:45.462 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:24:45.463 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243085 2023-10-14T00:24:45.463 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:24:45.464 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/583246470 ]] 2023-10-14T00:24:45.464 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/583246470 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\8\3\2\4\6\4\7\0 ]] 2023-10-14T00:24:45.464 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:24:45.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:45.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:45.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:45.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.501+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:45.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:45.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:45.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:45.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:45.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:45.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.753+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:45.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.753+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:45.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:45.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:46.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.101+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:46.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:46.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:46.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:46.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:46.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:46.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.141+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:46.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.141+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:46.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:46.141+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:46.332 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:46.333+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:46.332 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:46.333+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:46.701 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:46.701+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:24:46.722 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:46.725+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:47.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:47.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:47.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:47.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:47.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:47.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.577+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:47.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.577+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:47.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.593+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:47.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:47.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:47.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:47.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.673+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:47.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.845+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:47.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:47.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:47.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:47.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:47.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:47.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.869+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:47.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.869+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:47.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:47.873+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:48.159 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:48.161+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:24:48.166 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:48.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:49.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:49.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:49.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:49.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:49.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:49.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:49.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:49.113+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:51.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:51.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:51.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:51.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:51.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.349+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:51.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.349+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:51.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.349+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:51.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:51.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:51.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:51.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:51.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:51.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.513+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:51.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:51.513+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:52.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:52.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:52.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:52.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:52.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:52.933+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:52.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:53.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:53.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:53.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:53.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:53.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:53.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:53.041+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:53.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:53.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:53.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:53.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:53.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:53.373+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:53.724 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:53.725+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:55.479 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:24:55.479 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:24:55.481 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:24:55.509 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:24:55.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:55.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:55.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:55.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:55.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:55.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:55.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:55.757+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:55.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:55.757+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:55.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:55.809+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:56.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:56.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:56.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:56.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:56.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:56.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.749+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:56.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.749+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:56.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:56.749+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:56.932 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/583246470 2023-10-14T00:24:56.933 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:24:56.934 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243096 2023-10-14T00:24:56.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:24:56.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/583246470 ]] 2023-10-14T00:24:56.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/583246470 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\8\3\2\4\6\4\7\0 ]] 2023-10-14T00:24:56.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:24:57.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.361+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:57.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:57.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:57.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:57.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:57.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:57.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.389+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:57.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:57.389+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:24:59.375 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:24:59.377+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:24:59.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.805+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:24:59.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.805+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:24:59.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.821+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:24:59.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:24:59.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:24:59.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:24:59.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:24:59.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:24:59.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:24:59.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.833+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:24:59.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:24:59.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.041+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.041 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.041+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332ec00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.053+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.053+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253d100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.060 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.060 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.060 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c51c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb31efe80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2fbff00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3178280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.061+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.073 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.073+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.073 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.073+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332ec00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.073+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.073+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.073+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.073+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253d100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.077+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.077+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c51c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.101+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.101+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb31efe80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2fbff00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.117+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3178280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.117+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.117+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.122 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.121+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.122 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.121+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.129+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:25:00.129+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:25:00.492 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:00.493+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:25:00.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:00.509+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:02.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:02.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:02.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:02.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:02.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:02.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:03.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:03.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:03.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.005+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:03.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.005+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:03.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:03.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:03.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:03.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:03.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:03.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.109+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:03.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.109+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:03.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.137+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:03.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:03.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:03.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:03.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.277+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:03.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.277+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:03.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:03.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:03.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:03.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:03.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:03.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.485+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:03.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:03.489+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:04.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:04.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:04.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:04.789+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:06.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:06.573+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:25:06.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:06.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:06.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:06.585+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:06.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:06.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:06.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:06.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:06.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:06.609+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:06.936 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:25:06.936 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:25:06.946 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:25:06.960 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:25:08.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:08.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:08.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:08.386 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/583246470 2023-10-14T00:25:08.386 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:25:08.387 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243108 2023-10-14T00:25:08.388 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:25:08.388 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/583246470 ]] 2023-10-14T00:25:08.388 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/583246470 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\8\3\2\4\6\4\7\0 ]] 2023-10-14T00:25:08.388 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:25:08.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.409+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:08.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:08.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:08.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:08.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:08.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.801+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:08.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.801+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:08.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:08.801+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:08.874 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:08.873+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:09.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:09.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:09.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:09.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:09.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:09.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:09.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:09.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:09.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:09.041+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:09.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:09.041+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:09.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:09.049+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:10.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.077+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:10.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:10.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:10.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:10.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:10.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:10.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.125+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:10.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.125+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:10.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.141+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:10.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.557+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:10.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:10.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:10.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:10.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:10.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:10.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.613+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:10.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.613+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:10.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:10.613+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:11.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:11.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:11.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:11.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:11.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:11.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.817+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:11.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.817+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:11.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:11.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:12.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:12.820+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:25:12.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:12.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:12.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:12.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:12.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:12.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:12.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:12.888+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:13.025 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:13.024+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:13.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:13.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.292+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:13.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:13.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:13.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:13.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:13.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:13.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.316+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:13.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.316+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:13.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.316+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:13.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:13.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:13.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:13.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:13.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:13.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.628+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:13.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.628+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:13.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:13.636+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:14.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:14.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:14.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:14.304+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:15.457 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:15.456+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:15.518 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:15.516+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:16.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:16.400+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:16.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:16.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:16.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:16.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:16.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:16.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:16.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:16.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:16.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:16.436+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:17.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.168+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:17.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:17.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:17.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:17.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:17.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:17.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:17.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.224+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:17.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.228+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:17.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:17.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:17.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:17.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.232+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:17.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.232+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:17.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:17.232+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:18.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:18.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:18.389 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:25:18.390 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:25:18.393 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:25:18.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:18.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:18.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:18.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:18.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:18.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:18.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:18.404+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:18.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:18.404+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:18.409 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:25:19.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:19.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:19.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:19.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:19.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:19.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.292+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:19.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.292+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:19.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.304+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:19.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/583246470 2023-10-14T00:25:19.782 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:25:19.787 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243119 2023-10-14T00:25:19.788 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:25:19.788 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/583246470 ]] 2023-10-14T00:25:19.788 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/583246470 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\8\3\2\4\6\4\7\0 ]] 2023-10-14T00:25:19.788 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:25:19.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:19.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:19.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:19.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:19.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.896+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:19.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.896+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:19.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:19.896+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:20.055 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:20.056+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:21.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:21.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:21.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:21.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:21.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:21.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:22.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:22.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:22.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:22.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:22.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:22.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:22.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:22.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:22.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:22.624+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:22.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:22.624+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:22.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:22.636+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:23.902 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:23.900+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:25:23.914 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:23.916+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:25.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.428+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:25.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:25.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:25.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:25.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:25.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:25.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.460+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:25.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.460+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:25.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:25.460+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:26.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:26.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:26.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:26.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:26.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:26.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:26.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:26.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:26.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:26.996+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:27.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:27.000+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:27.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:27.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:27.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:27.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:27.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:27.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:27.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:27.128+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:27.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:27.128+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:27.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:27.148+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:28.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.092+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:28.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:28.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:28.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:28.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:28.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:28.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.160+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:28.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.160+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:28.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:28.160+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:29.133 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:29.132+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:25:29.181 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:29.180+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:29.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.240+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:29.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:29.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:29.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:29.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:29.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:29.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.392+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:29.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.392+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:29.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.404+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:29.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.428+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:25:29.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.428+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:25:29.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.432+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:29.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:29.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:29.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:29.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:29.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:29.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.492+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:29.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.492+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:29.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.496+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:29.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.584+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:29.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:29.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:29.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:29.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:29.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:29.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.624+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:29.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.624+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:29.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.632+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:29.799 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:25:29.800 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:25:29.806 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:25:29.813 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:25:29.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:29.960+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:31.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:31.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:31.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:31.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:31.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.144+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:31.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.144+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:31.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.156+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:31.267 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:25:31.268 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:25:31.269 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243131 2023-10-14T00:25:31.269 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:25:31.269 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/870020856 ]] 2023-10-14T00:25:31.269 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/870020856 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\5\8\3\2\4\6\4\7\0 ]] 2023-10-14T00:25:31.269 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/870020856 2023-10-14T00:25:31.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:31.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:31.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:31.432+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:32.030 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:32.032+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:33.310 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/870020856 until 2023-10-14T01:25:32.606480+0000 (3600 sec) 2023-10-14T00:25:33.341 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:25:33.355 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/870020856 2023-10-14T00:25:34.132 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:34.132+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:34.243 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:34.244+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:34.349 INFO:tasks.workunit.client.0.smithi183.stderr:listed 23 entries 2023-10-14T00:25:34.386 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:25:34.386 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:25:34.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:34.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:34.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:34.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:34.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:34.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.804+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:34.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.804+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:34.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:34.808+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:35.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.616+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:25:35.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.616+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:25:35.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.616+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:35.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:35.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:35.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:35.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:35.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:35.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.660+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:35.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.660+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:35.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:35.664+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:37.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:37.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:37.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:37.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:37.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.080+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:37.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.080+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:37.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.084+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:37.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.416+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:25:37.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.428+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:25:37.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.452+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:37.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:37.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:37.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:37.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:37.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:37.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:37.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.476+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:37.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.484+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:37.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:37.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.764+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:37.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:37.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:37.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:37.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:37.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.964+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:37.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.964+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:37.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:37.972+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:43.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:43.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:43.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:43.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:43.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:43.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:43.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:43.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:43.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:43.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:43.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:43.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:43.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:43.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:44.290 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:44.292+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:25:44.312 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:44.312+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:44.389 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:25:44.389 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:25:44.397 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:25:44.408 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:25:44.596 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:44.596+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:45.778 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:25:45.781 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:25:45.782 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243145 2023-10-14T00:25:45.782 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:25:45.782 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/870020856 ]] 2023-10-14T00:25:45.783 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/870020856 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\8\7\0\0\2\0\8\5\6 ]] 2023-10-14T00:25:45.783 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:25:46.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:46.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:46.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:46.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:46.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:46.844+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:46.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:46.848+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:46.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:46.892+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:48.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:48.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:48.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:48.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:48.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:48.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.488+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:48.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:48.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:48.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:48.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:48.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:48.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:48.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:48.596+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:50.586 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:50.592+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:51.586 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:51.588+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:52.261 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:52.264+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:52.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:52.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.724+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.724+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:52.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.724+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:52.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:52.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:53.433 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:53.436+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:54.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.188+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:54.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:54.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:54.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:54.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:54.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:54.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.244+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:54.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.244+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:54.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.244+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:54.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:54.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:54.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:54.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:54.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.572+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:54.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.624+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:54.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:54.628+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:55.784 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:25:55.784 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:25:55.788 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:25:55.793 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:25:55.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.816+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:55.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:55.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:55.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:55.933 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:55.936+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:25:55.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:55.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:55.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.936+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:55.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.936+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:55.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:55.936+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:55.963 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:55.968+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:55.963 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:55.968+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:56.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:56.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:56.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:56.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:56.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:56.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.236+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:56.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.236+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:56.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.236+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:56.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:56.572+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:57.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.020+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:57.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:57.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:57.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:57.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:57.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.136+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:57.133 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:25:57.133 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:25:57.134 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243157 2023-10-14T00:25:57.134 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:25:57.134 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/870020856 ]] 2023-10-14T00:25:57.135 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/870020856 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\8\7\0\0\2\0\8\5\6 ]] 2023-10-14T00:25:57.135 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:25:57.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.136+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:57.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.144+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:57.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.164+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:57.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:25:57.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:57.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:57.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:57.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:57.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.204+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:57.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.204+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:57.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.208+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:57.281 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:57.284+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:25:57.309 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:57.312+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:57.314 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:57.320+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:57.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:57.892+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:58.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.028+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:58.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:58.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:58.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:58.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:58.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.068+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:58.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.068+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:58.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.068+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:58.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.444+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:25:58.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:58.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:58.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:58.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:58.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:58.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:58.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:58.581 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:58.584+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:25:58.584 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:25:58.588+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:25:58.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:25:58.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:25:58.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:25:58.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:25:58.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.712+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:25:58.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.712+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:25:58.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:58.728+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:25:59.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:25:59.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:00.074 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffbf00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c51480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb253df80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d28e80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.077 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.080+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3258000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.086 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.092+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.086 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.092+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb1e1bc00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.108+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.105 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.108+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c3b300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.116+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.114 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.116+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2931900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.124+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.124+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.124+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.124+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.124+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e4b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.124+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e5e80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.122 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.123 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e700 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.124 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.124 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2930b80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.128+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352eb00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.132+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.132+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352ed00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.136+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.136+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352ef00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.142 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.148+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.143 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.148+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352f900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.143 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.148+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.143 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:26:00.148+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352f780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:26:00.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:00.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:00.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:00.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:00.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:00.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:00.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:00.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.436+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:00.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.504+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:00.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:00.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:00.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:00.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:00.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:00.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:00.960 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:00.963+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:26:00.974 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:00.975+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:01.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:01.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:01.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:01.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:01.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:01.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.735+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:01.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.735+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:01.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:01.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.751+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:26:01.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.787+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:01.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:01.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:01.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:01.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:01.799+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:04.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.779+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:04.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:04.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:04.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:04.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:04.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:04.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.863+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:04.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.863+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:04.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:04.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:06.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:06.367+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:06.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:06.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:06.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:06.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:06.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:06.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:06.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:06.403+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:07.088 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:07.091+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:26:07.136 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:26:07.136 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:26:07.137 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:26:07.154 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:26:08.411 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:26:08.412 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:26:08.415 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243168 2023-10-14T00:26:08.415 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:26:08.415 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/870020856 ]] 2023-10-14T00:26:08.416 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/870020856 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\8\7\0\0\2\0\8\5\6 ]] 2023-10-14T00:26:08.416 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:26:09.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.659+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:09.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:09.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:09.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:09.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:09.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:09.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.687+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:09.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.687+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:09.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:09.687+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:11.649 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:11.651+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:26:11.671 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:11.675+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:26:11.675 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:11.679+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:26:11.703 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:11.707+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:12.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:12.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:12.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:12.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:12.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:12.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:12.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:12.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:12.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:12.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:12.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:12.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:13.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.027+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:26:13.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.027+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:26:13.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.031+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:13.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:13.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:13.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:13.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:13.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:13.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.079+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:13.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.079+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:13.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:13.083+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:16.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.647+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:16.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:16.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:16.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:16.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:16.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.667+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:16.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.667+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:16.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:16.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:18.435 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:26:18.443 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:26:18.447 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:26:18.465 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:26:18.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:18.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:18.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:18.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:18.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:18.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.883+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:18.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.883+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:18.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:18.883+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:19.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:19.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:19.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:19.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:19.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:19.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:19.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:19.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:19.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:19.839+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:19.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:19.839+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:19.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:19.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:19.909 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:26:19.910 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:26:19.910 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243179 2023-10-14T00:26:19.911 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:26:19.911 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/870020856 ]] 2023-10-14T00:26:19.911 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/870020856 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\8\7\0\0\2\0\8\5\6 ]] 2023-10-14T00:26:19.911 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:26:20.422 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:20.427+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:26:20.442 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:20.447+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:26:21.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:21.819+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:21.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:21.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:21.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:21.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:21.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:21.867+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:22.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:22.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:22.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:22.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:22.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:22.099+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:24.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:24.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:24.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:24.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:24.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:24.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.691+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:24.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.691+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:24.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.703+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:24.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.795+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:24.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:24.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:24.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:24.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:24.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:24.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.843+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:24.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.843+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:24.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.843+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:24.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:24.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:24.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:24.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:24.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.863+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:24.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:24.883+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:26.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.091+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:26.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:26.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:26.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:26.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:26.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:26.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.143+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:26.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.143+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:26.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:26.143+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:26.403 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:26.407+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:27.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.395+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:27.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:27.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:27.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:27.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:27.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:27.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.455+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:27.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.455+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:27.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:27.455+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:28.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.199+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:28.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:28.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:28.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:28.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:28.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:28.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.267+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:28.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.267+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:28.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.267+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:28.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:28.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:28.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:28.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:28.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.743+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:28.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.743+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:28.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.879+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:28.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:28.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:28.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:28.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:28.907+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:29.213 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:29.215+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:29.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:29.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:29.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:29.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.375+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:29.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.375+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:29.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:29.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:29.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:29.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:29.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.443+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:29.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:29.443+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:29.938 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:26:29.941 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:26:29.973 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:26:29.990 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:26:30.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:30.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:30.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:30.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:30.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:30.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.791+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:30.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.791+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:30.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:30.791+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:31.243 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:26:31.249 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:26:31.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243191 2023-10-14T00:26:31.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:26:31.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/870020856 ]] 2023-10-14T00:26:31.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/870020856 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\8\7\0\0\2\0\8\5\6 ]] 2023-10-14T00:26:31.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:26:31.902 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:31.907+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:26:31.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:31.947+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:26:31.961 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:31.963+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:35.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:35.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:35.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:35.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:35.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:35.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:35.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:35.055+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:36.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:36.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:36.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:36.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:36.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:36.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.199+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:36.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.199+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:36.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:36.207+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:38.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:38.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:38.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:38.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:38.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:38.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.859+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:38.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.859+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:38.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:38.879+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:40.501 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:40.503+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:26:40.501 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:40.503+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:41.257 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:26:41.257 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:26:41.258 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:26:41.281 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:26:42.722 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/870020856 2023-10-14T00:26:42.723 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:26:42.723 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243202 2023-10-14T00:26:42.724 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:26:42.724 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/870020856 ]] 2023-10-14T00:26:42.724 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/870020856 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\8\7\0\0\2\0\8\5\6 ]] 2023-10-14T00:26:42.724 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:26:44.080 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:44.083+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:44.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:44.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:44.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:44.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.483+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:44.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.483+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:44.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.503+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:44.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:44.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:44.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:44.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:44.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:44.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.699+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:44.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.699+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:44.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:44.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:44.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:44.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:44.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:44.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:44.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.819+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:44.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.819+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:44.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:44.823+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:45.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.611+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:45.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:45.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:45.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:45.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:45.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.675+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:45.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.675+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:45.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:45.687+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:46.362 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:46.367+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:47.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.283+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:47.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:47.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:47.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:47.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:47.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:47.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.299+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:47.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.299+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:47.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.299+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:47.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.671+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:47.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:47.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:47.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:47.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:47.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:47.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.719+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:47.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.719+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:47.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:47.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:48.746 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:26:48.747+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:26:49.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:49.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:49.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:49.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:49.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:49.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:49.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:49.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:49.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:49.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:49.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:49.799+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:50.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:50.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:50.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:50.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:50.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:50.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:50.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:50.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:50.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:50.279+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:51.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:51.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:51.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:51.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:51.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:51.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.327+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:51.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.327+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:51.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:51.327+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:51.435 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:51.439+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:51.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:51.875+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:52.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:52.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:52.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.542+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:52.725 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:26:52.729 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:26:52.741 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:26:52.773 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:26:52.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:52.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:52.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:52.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:52.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.990+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:52.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.990+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:52.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:52.990+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:54.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:54.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:54.222 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4019572365 2023-10-14T00:26:54.235 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:26:54.238 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243214 2023-10-14T00:26:54.238 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:26:54.238 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4019572365 ]] 2023-10-14T00:26:54.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4019572365 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\8\7\0\0\2\0\8\5\6 ]] 2023-10-14T00:26:54.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/4019572365 2023-10-14T00:26:56.258 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/4019572365 until 2023-10-14T01:26:55.598832+0000 (3600 sec) 2023-10-14T00:26:56.332 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:26:56.340 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/4019572365 2023-10-14T00:26:56.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:56.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:56.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:56.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:56.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.410+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:56.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.410+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:56.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.410+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:56.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:56.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:56.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:56.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:56.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.866+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:56.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.866+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:56.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.870+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:56.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:56.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:56.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:57.482 INFO:tasks.workunit.client.0.smithi183.stderr:listed 24 entries 2023-10-14T00:26:57.537 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/4019572365 2023-10-14T00:26:57.538 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:26:57.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.558+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:57.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:57.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:57.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.642+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.642+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.642+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:57.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.730+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:26:57.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:26:57.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:57.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:57.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:57.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.802+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:57.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.802+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:57.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:57.810+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:57.917 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:57.918+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:26:58.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:58.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:26:58.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:58.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:26:58.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:58.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:26:58.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:58.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:26:58.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:58.510+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:26:58.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:58.510+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:26:58.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:26:58.530+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:26:59.687 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:26:59.690+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.002+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.002+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c3a800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.014 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.018+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.018+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2c50280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.018+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.016 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.018+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.033 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.038+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.034 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.038+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3257d80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.038+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.038+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d92b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.049 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.050+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.049 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.050+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb325b580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.078+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:27:00.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.078+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:27:00.077 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.078+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.078 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.078+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332f200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:00.108 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:00.110+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:27:00.122 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:00.126+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:27:00.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:00.129 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.134+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.134+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.136 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.138+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.136 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.138+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb31efe80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:00.149 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:00.154+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:00.154 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.158+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.154 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.158+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2ffbc00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.155 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.158+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.155 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.158+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332fd80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:00.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:00.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.158+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:00.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.158+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:00.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:00.158+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:00.174 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.178+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.174 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.178+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332ec80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.190 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.194+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.190 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.194+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3528900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.194 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.198+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.194 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.198+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3528b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.238 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.238+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.238 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.238+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3528d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.241 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.242+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.241 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.242+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3528f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.251 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.254+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.251 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.254+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3529100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.266 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.270+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.267 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.270+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3529300 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.267 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.270+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.267 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.270+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3529d00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.268 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.270+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:00.268 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:27:00.270+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3529b80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:27:01.404 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:01.406+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:27:01.418 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:01.422+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:02.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:02.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:02.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:02.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:02.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:02.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.242+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:02.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.242+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:02.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.242+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:02.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:02.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:02.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:02.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:02.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.682+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:02.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.682+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:02.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:02.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:02.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:02.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:02.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.994+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:02.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:02.994+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:03.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:03.002+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:03.567 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:03.570+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:27:03.582 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:03.586+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:04.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.138+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:27:04.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.138+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:27:04.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.158+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:04.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:04.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:04.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:04.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:04.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:04.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.222+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:04.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.234+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:04.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:04.250+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:06.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:06.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:06.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:06.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:06.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:06.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.626+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:06.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.626+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:06.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.634+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:06.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.894+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:06.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:06.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:06.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:06.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:06.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:06.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.942+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:06.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:06.942+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:07.548 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:27:07.549 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:27:07.553 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:27:07.562 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:27:08.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.358+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:08.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:08.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:08.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.410+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:08.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4019572365 2023-10-14T00:27:08.876 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:27:08.877 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243228 2023-10-14T00:27:08.878 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:27:08.878 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4019572365 ]] 2023-10-14T00:27:08.878 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4019572365 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\1\9\5\7\2\3\6\5 ]] 2023-10-14T00:27:08.878 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:27:08.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.918+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:27:08.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.918+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:27:08.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.926+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:08.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:08.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:08.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:08.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:08.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:08.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.982+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:08.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:08.982+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:09.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.002+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:09.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:09.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:09.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:09.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:09.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:09.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:09.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:09.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:09.710+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:10.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.962+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:10.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:10.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:10.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:10.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:10.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:10.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.994+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:10.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:10.994+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:11.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.002+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:11.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.130+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:11.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:11.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:11.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:11.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:11.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.138+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:11.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.138+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:11.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:11.142+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:12.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.646+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:12.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:12.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:12.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:12.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:12.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:12.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.686+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:12.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.686+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:12.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.686+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:12.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.978+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:12.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:12.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:12.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:13.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:13.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.002+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:13.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:13.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:13.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:13.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:13.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.478+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:13.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.478+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:13.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.478+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:13.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.870+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:13.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:13.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:13.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:13.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:13.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:13.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.946+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:13.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.946+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:13.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.946+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:13.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:13.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:14.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:14.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:14.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.042+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:14.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:14.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:14.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:14.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.582+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:14.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.582+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:14.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:14.582+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:15.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:15.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:15.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:15.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:15.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:15.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:15.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:15.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:15.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:15.994+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:16.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:16.002+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:16.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:16.014+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:16.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:16.390+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:16.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:16.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:16.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:16.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:16.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:16.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:16.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:16.434+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:17.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:17.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:17.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:17.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:17.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:17.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:17.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:17.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:17.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:17.066+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:17.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:17.066+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:17.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:17.066+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:18.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:18.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:18.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:18.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:18.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:18.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.414+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:18.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.442+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:18.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:18.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:18.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:18.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:18.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:18.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.790+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:18.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.790+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:18.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:18.798+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:18.880 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:27:18.881 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:27:18.881 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:27:18.882 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:27:19.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:19.430+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:19.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:19.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:19.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:19.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:19.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:19.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:19.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:19.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:19.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:19.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:19.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:19.486+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:20.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4019572365 2023-10-14T00:27:20.284 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:27:20.291 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243240 2023-10-14T00:27:20.292 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:27:20.292 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4019572365 ]] 2023-10-14T00:27:20.293 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4019572365 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\1\9\5\7\2\3\6\5 ]] 2023-10-14T00:27:20.293 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:27:20.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:20.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:20.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:20.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:20.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:20.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:20.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:20.462+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:20.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:20.462+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:21.140 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:21.142+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:27:21.140 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:21.142+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:23.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:23.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:23.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.074+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:23.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:23.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:23.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:23.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.594+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:23.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:23.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:23.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:23.658+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:24.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:24.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:24.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:24.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:24.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:24.606+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:25.020 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:25.022+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:27:25.020 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:25.022+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:25.198 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:25.202+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:27:25.216 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:25.218+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:27.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:27.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:27.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:27.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:27.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:27.058+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:27.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:27.406+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:27:27.406 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:27.410+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:28.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.678+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:28.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:28.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:28.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:28.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:28.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:28.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.742+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:28.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.742+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:28.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:28.742+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:29.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.590+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:27:29.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.590+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:27:29.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.618+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:29.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:29.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:29.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:29.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:29.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:29.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.662+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:29.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.662+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:29.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:29.662+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:30.301 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:27:30.306 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:27:30.354 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:27:30.355 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:27:30.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.914+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:30.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:30.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:30.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:30.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:30.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:30.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.950+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:30.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.950+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:30.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:30.954+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:31.382 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:31.386+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:31.711 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4019572365 2023-10-14T00:27:31.713 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:27:31.726 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243251 2023-10-14T00:27:31.726 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:27:31.726 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4019572365 ]] 2023-10-14T00:27:31.726 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4019572365 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\1\9\5\7\2\3\6\5 ]] 2023-10-14T00:27:31.727 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:27:33.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:33.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:33.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:33.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:33.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.078+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:33.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.078+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:33.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.078+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:33.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:33.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.334+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:33.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.630+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:33.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:33.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:33.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:33.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:33.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:33.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.714+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:33.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.714+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:33.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.714+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:33.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:33.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:33.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:33.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:33.842+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:34.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:34.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:34.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:34.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:34.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:34.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.798+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:34.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.798+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:34.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.806+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:34.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:34.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.954+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:34.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.954+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:34.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.954+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:34.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.954+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:34.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:34.954+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:35.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:35.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:35.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:35.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:35.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:35.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:35.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.614+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:35.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:35.630+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:36.611 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:36.614+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:40.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:40.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:40.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:40.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:41.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:41.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:41.042+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:41.042+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:41.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:41.042+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:41.737 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:27:41.737 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:27:41.737 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:27:41.739 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:27:42.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:42.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:42.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:42.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:42.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:42.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.750+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:42.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.750+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:42.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:42.798+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:42.825 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4019572365 2023-10-14T00:27:42.826 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:27:42.826 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243262 2023-10-14T00:27:42.826 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:27:42.827 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4019572365 ]] 2023-10-14T00:27:42.827 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4019572365 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\1\9\5\7\2\3\6\5 ]] 2023-10-14T00:27:42.827 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:27:42.975 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:42.978+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:43.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:43.090 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:27:43.094+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:27:43.102 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:27:43.102+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:27:43.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.342+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:43.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:43.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.382+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:43.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:43.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:43.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:43.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.386+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:43.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.386+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:43.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.386+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:43.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:43.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:43.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:43.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.750+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:43.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:43.750+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:45.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.382+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:45.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:45.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:45.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:45.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.394+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:45.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.394+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:45.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.394+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:45.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:45.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:45.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:45.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:45.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.882+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:45.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.898+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:45.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:45.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:46.254 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:46.258+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:47.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:47.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:47.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:47.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:47.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:47.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.273+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:47.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.273+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:47.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.301+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:47.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.325+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:27:47.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.337+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:47.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:47.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:47.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:47.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:47.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:47.369+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:50.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.137+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:50.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:50.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:50.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:50.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:50.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.153+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:50.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.153+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:50.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.153+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:50.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.561+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:50.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:50.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:50.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:50.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:50.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.581+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:50.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.581+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:50.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:50.589+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:51.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:51.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:51.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:51.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.085+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:51.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.085+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:51.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.089+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:51.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:51.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:51.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:51.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:51.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:51.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:52.828 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:27:52.829 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:27:52.835 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:27:52.851 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:27:52.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:52.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:52.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:52.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:52.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:52.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:52.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:52.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:52.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:52.973+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:52.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:52.973+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:52.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:52.973+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:53.458 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:53.461+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:27:53.472 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:53.473+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:27:53.478 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:53.481+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:54.299 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4019572365 2023-10-14T00:27:54.299 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:27:54.300 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243274 2023-10-14T00:27:54.300 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:27:54.300 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4019572365 ]] 2023-10-14T00:27:54.300 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4019572365 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\1\9\5\7\2\3\6\5 ]] 2023-10-14T00:27:54.301 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:27:54.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.369+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:54.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:54.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:54.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:54.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:54.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:54.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:54.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:54.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:55.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.209+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:55.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:55.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:55.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:55.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:55.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.281+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:55.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.281+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:55.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:55.313+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:57.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.677+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:27:57.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:27:57.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:57.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.769+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:57.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.769+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:57.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:57.785+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:27:59.289 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:27:59.285+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:27:59.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:59.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:27:59.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:59.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:27:59.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:59.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:27:59.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:59.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:27:59.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:59.721+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:27:59.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:59.721+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:27:59.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:27:59.721+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:00.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:00.977+0000 7faeed24b640 -1 reset not still connected to 0x556f4925c0d0 2023-10-14T00:28:00.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:00.981+0000 7faeed24b640 -1 reset not still connected to 0x556f4925c5b0 2023-10-14T00:28:00.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:00.981+0000 7faeed24b640 -1 reset not still connected to 0x556f4925d2b0 2023-10-14T00:28:01.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:01.005+0000 7faeed24b640 -1 reset not still connected to 0x556f4925dc70 2023-10-14T00:28:01.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:01.017+0000 7faeed24b640 -1 reset not still connected to 0x556f4925de10 2023-10-14T00:28:01.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:01.017+0000 7faeed24b640 -1 reset not still connected to 0x556f4925dee0 2023-10-14T00:28:02.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:02.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:02.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.561+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:02.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:02.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:02.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.581+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:02.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.581+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:02.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.589+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:02.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.637+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:02.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:02.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:02.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:02.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:02.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.673+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:02.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.797+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:02.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:02.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:02.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:02.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:02.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:02.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.853+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:02.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.853+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:02.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:02.853+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:02.915 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:02.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:28:02.953 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:02.953+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:04.301 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:28:04.302 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:28:04.302 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:28:04.309 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:28:05.279 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4068544102 2023-10-14T00:28:05.279 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:28:05.279 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243285 2023-10-14T00:28:05.280 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:28:05.280 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4068544102 ]] 2023-10-14T00:28:05.280 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4068544102 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\1\9\5\7\2\3\6\5 ]] 2023-10-14T00:28:05.280 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/4068544102 2023-10-14T00:28:05.779 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:05.781+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:28:05.779 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:05.781+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:06.393 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.397+0000 7f660e1ef640 -1 librbd::ImageWatcher: 0x55ccb32dd500 image watch failed: 94337658388480, (107) Transport endpoint is not connected 2023-10-14T00:28:06.394 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.397+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb32dd500 handle_error: handle=94337658388480: (107) Transport endpoint is not connected 2023-10-14T00:28:06.395 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.397+0000 7f660e1ef640 -1 librbd::watcher::RewatchRequest: 0x55ccb29205a0 handle_unwatch client blocklisted 2023-10-14T00:28:06.395 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.397+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb32dd500 handle_rewatch: client blocklisted 2023-10-14T00:28:06.395 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.397+0000 7f660e1ef640 -1 librbd::ManagedLock: 0x55ccb1efbd58 send_acquire_lock: watcher not registered - client blocklisted 2023-10-14T00:28:06.395 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.397+0000 7f660e1ef640 -1 librbd::mirror::snapshot::CreatePrimaryRequest: 0x55ccb3210000 handle_create_snapshot: failed to create mirror snapshot: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:28:06.395 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.397+0000 7f660e1ef640 -1 librbd::io::AioCompletion: 0x55ccb36a0f20 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:28:06.534 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::ImageWatcher: 0x55ccb32ddb00 image watch failed: 94337654639616, (107) Transport endpoint is not connected 2023-10-14T00:28:06.534 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb32ddb00 handle_error: handle=94337654639616: (107) Transport endpoint is not connected 2023-10-14T00:28:06.534 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::managed_lock::GetLockerRequest: 0x55ccb2673030 handle_get_lockers: failed to retrieve lockers: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:28:06.534 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::managed_lock::AcquireRequest: 0x55ccb369bd10 handle_get_locker: failed to retrieve lockers: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:28:06.535 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::ManagedLock: 0x55ccb1d4e698 handle_acquire_lock: failed to acquire exclusive lock: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:28:06.535 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::mirror::snapshot::CreatePrimaryRequest: 0x55ccb3210dd0 handle_create_snapshot: failed to create mirror snapshot: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:28:06.535 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::io::AioCompletion: 0x55ccb317b4a0 fail: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:28:06.535 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::watcher::RewatchRequest: 0x55ccb28a2e10 handle_unwatch client blocklisted 2023-10-14T00:28:06.535 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:28:06.533+0000 7f660e1ef640 -1 librbd::Watcher: 0x55ccb32ddb00 handle_rewatch: client blocklisted 2023-10-14T00:28:07.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:07.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:07.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:07.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:07.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:07.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:07.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:07.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:07.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:07.321+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:07.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:07.321+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:07.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:07.321+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:07.333 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/4068544102 until 2023-10-14T01:28:06.522815+0000 (3600 sec) 2023-10-14T00:28:07.362 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:28:07.362 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/4068544102 2023-10-14T00:28:08.104 INFO:tasks.workunit.client.0.smithi183.stderr:listed 25 entries 2023-10-14T00:28:08.139 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/4068544102 2023-10-14T00:28:08.140 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:28:12.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:12.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:12.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:12.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:12.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:12.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:12.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:12.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:12.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:12.305+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:12.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:12.305+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:12.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:12.341+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:14.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:14.569+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:14.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:14.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:14.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:14.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:14.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:14.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:14.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:14.665+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:14.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:14.665+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:14.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:14.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:15.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:15.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:16.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:16.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:16.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:16.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:16.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:16.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:16.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:16.073+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:16.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:16.073+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:16.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:16.089+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:17.081 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:17.085+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:28:17.107 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:17.109+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:18.142 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:28:18.142 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:28:18.142 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:28:18.143 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:28:19.017 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4068544102 2023-10-14T00:28:19.017 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:28:19.018 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243299 2023-10-14T00:28:19.018 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:28:19.018 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4068544102 ]] 2023-10-14T00:28:19.019 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4068544102 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\8\5\4\4\1\0\2 ]] 2023-10-14T00:28:19.019 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:28:20.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:20.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:20.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:20.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:20.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:20.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:20.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:20.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:20.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:20.213+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:20.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:20.213+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:20.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:20.249+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:21.688 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:21.689+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:28:21.702 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:21.705+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:23.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:23.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:23.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:23.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:23.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:23.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:23.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:23.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:23.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:23.209+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:23.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:23.209+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:23.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:23.249+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:24.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.560+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:28:24.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:24.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:24.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:24.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:24.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:24.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.744+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:24.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.744+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:24.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:24.744+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:27.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:27.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:27.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:27.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:27.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:27.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:27.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:27.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:27.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:27.328+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:27.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:27.328+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:27.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:27.368+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:29.023 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:28:29.024 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:28:29.025 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:28:29.029 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:28:29.031 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:29.032+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:28:29.058 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:29.060+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:29.153 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:29.156+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:30.216 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4068544102 2023-10-14T00:28:30.217 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:28:30.217 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243310 2023-10-14T00:28:30.217 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:28:30.217 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4068544102 ]] 2023-10-14T00:28:30.218 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4068544102 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\8\5\4\4\1\0\2 ]] 2023-10-14T00:28:30.218 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:28:31.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.516+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:31.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:31.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:31.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:31.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:31.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:31.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.636+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:31.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.636+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:31.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:31.636+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:31.705 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:31.700+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:33.459 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:33.456+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:28:33.635 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:33.636+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:40.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.164+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:40.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:40.221 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:28:40.225 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:28:40.234 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:28:40.237 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:28:40.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:40.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:40.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:40.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:40.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.260+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:40.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.260+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:40.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:40.260+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:41.593 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4068544102 2023-10-14T00:28:41.594 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:28:41.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243321 2023-10-14T00:28:41.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:28:41.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4068544102 ]] 2023-10-14T00:28:41.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4068544102 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\8\5\4\4\1\0\2 ]] 2023-10-14T00:28:41.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:28:42.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.196+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:42.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.748+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:42.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:42.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:42.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:42.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:42.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:42.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:42.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:42.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.792+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:42.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:42.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:42.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:42.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:42.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.928+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:42.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:42.928+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:43.628 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:43.632+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:44.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.580+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:44.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:44.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:44.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:44.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:44.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:44.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.688+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:44.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.688+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:44.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:44.720+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:45.671 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:28:45.676+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:28:45.671 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:28:45.676+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:28:45.678 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:28:45.684+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T00:28:45.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:45.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:45.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:45.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:45.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:45.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:45.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:45.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:45.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:45.740+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:45.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:45.740+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:45.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:45.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:46.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:46.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:46.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:46.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:46.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:46.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:46.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.340+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:46.345 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:46.348+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:28:46.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:46.348+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:46.370 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:46.376+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:28:46.375 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:46.380+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:47.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:47.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:47.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:47.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:47.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:47.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:47.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:47.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:47.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:47.916+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:47.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:47.916+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:48.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.552+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:28:48.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.564+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:48.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:48.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:48.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:48.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:48.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:48.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.596+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:48.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.596+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:48.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.596+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:48.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:48.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:48.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:48.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:48.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:48.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.988+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:48.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.988+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:48.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:48.988+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:49.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:49.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:49.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:49.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:49.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:49.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:49.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:49.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:49.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:49.439+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:49.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:49.439+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:50.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:50.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:50.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:50.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.251+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:50.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.275+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:50.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:50.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:50.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:50.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:50.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:50.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.523+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:50.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.523+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:50.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:50.523+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:51.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.155+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:28:51.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:51.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:51.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:51.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:51.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.279+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:51.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.279+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:51.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.307+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:51.611 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:28:51.612 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:28:51.613 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:28:51.626 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:28:51.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.899+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:51.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:51.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:51.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:51.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:51.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:51.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.927+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:51.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.927+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:51.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:51.963+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:52.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:52.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:52.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:52.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:52.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:52.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:52.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:52.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:52.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:52.107+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:52.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:52.107+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:52.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:52.107+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:52.394 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:52.395+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:28:52.409 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:52.411+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:53.040 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4068544102 2023-10-14T00:28:53.040 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:28:53.041 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243333 2023-10-14T00:28:53.041 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:28:53.041 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4068544102 ]] 2023-10-14T00:28:53.041 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4068544102 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\8\5\4\4\1\0\2 ]] 2023-10-14T00:28:53.042 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:28:54.424 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:54.427+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:28:54.436 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:28:54.439+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:28:55.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:55.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:55.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:55.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:55.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.335+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:55.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.335+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:55.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.335+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:55.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:55.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:55.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.651+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:55.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:55.651+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:56.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:56.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:56.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:56.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:56.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:56.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:56.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:56.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:56.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:56.351+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:56.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:56.351+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:56.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:56.351+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:59.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.039+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:28:59.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.051+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:28:59.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.071+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:59.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:59.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:59.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:59.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:59.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:59.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.155+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:59.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.155+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:59.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.155+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:59.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.435+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:28:59.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:28:59.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:59.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:59.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:59.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:59.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.479+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:28:59.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.479+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:28:59.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.479+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:28:59.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:28:59.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:28:59.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:28:59.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:28:59.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:28:59.763+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:02.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:02.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:02.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:02.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:02.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:02.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.543+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:02.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.543+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:02.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.543+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:02.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.799+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:29:02.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.799+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:29:02.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.835+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:02.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:02.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:02.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:02.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:02.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:02.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.867+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:02.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.867+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:02.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:02.871+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:03.044 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:29:03.045 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:29:03.048 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:29:03.051 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:29:03.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.879+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:03.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:03.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:03.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:03.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:03.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.919+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:03.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.919+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:03.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:03.919+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:04.014 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:04.019+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:29:04.014 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:04.019+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:29:04.014 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:04.019+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:04.015 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:04.019+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:04.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:04.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:04.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:04.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:04.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:04.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.323+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:04.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.323+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:04.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:04.327+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:04.641 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4068544102 2023-10-14T00:29:04.641 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:29:04.642 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243344 2023-10-14T00:29:04.642 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:29:04.642 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4068544102 ]] 2023-10-14T00:29:04.643 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4068544102 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\8\5\4\4\1\0\2 ]] 2023-10-14T00:29:04.643 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:29:05.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:05.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:05.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:05.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:05.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:05.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:05.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:05.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:05.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:05.435+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:05.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:05.435+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:05.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:05.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:06.589 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:06.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:07.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:07.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:07.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:07.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:07.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:07.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.203+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:07.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.203+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:07.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.215+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:07.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:07.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:07.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:07.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:07.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:07.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.367+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:07.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.367+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:07.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.403+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:07.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:07.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:07.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:07.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:07.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:07.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.635+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:07.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.635+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:07.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:07.635+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:08.310 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:08.315+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:29:08.315 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:08.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:08.348 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:08.351+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:09.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.643+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:09.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:09.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:09.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:09.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:09.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.675+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:09.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.675+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:09.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:09.675+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:11.406 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:11.407+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:11.426 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:11.427+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:13.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:13.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:13.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:13.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:13.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.455+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:13.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.455+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:13.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.463+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:13.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:13.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:13.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:13.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:13.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.875+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:13.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.907+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:29:13.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:13.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:13.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:13.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:13.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:13.951+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:14.646 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:29:14.646 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:29:14.648 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:29:14.648 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:29:14.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.731+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:14.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:14.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:14.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:14.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.919+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:14.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:14.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:14.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:14.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:14.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:14.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.963+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:14.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.963+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:14.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:14.963+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:16.212 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:16.210+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:29:16.220 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:16.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:16.246 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/981386002 2023-10-14T00:29:16.255 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:29:16.260 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243356 2023-10-14T00:29:16.260 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:29:16.260 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/981386002 ]] 2023-10-14T00:29:16.260 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/981386002 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\8\5\4\4\1\0\2 ]] 2023-10-14T00:29:16.261 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/981386002 2023-10-14T00:29:17.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:17.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:17.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:17.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:17.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:17.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:17.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:17.054+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:17.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:17.082+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:18.378 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/981386002 until 2023-10-14T01:29:17.744994+0000 (3600 sec) 2023-10-14T00:29:18.425 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/981386002 2023-10-14T00:29:18.425 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:29:18.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:18.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:18.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:18.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:18.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:18.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:18.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:18.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:18.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:18.650+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:18.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:18.650+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:18.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:18.658+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:19.545 INFO:tasks.workunit.client.0.smithi183.stderr:listed 26 entries 2023-10-14T00:29:19.595 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/981386002 2023-10-14T00:29:19.596 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:29:20.238 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:20.242+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:20.626 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:20.626+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:21.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:21.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:21.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:21.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:21.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:21.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:21.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:21.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:21.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:21.570+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:21.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:21.590+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:21.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:21.590+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:23.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:23.966+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:23.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:23.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:24.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:24.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:24.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:24.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:24.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:24.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:24.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:24.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:24.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:24.022+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:24.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:24.022+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:24.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:24.022+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:25.289 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:25.286+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:29:25.292 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:25.294+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:29:25.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:25.302+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:25.308 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:25.310+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:25.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:25.974+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:25.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:25.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:25.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:25.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:25.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:25.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:25.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:25.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:25.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:25.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:25.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:25.990+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:27.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:27.978+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:27.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:27.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:28.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:28.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:28.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:28.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:28.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:28.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:28.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:28.006+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:28.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:28.006+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:28.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:28.006+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:29.097 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:29.094+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:29:29.100 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:29.102+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:29.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:29.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:29.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:29.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:29.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:29.250+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:29.462 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:29.462+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:29.605 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:29:29.606 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:29:29.606 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:29:29.610 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:29:30.990 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/981386002 2023-10-14T00:29:30.990 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:29:30.991 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243370 2023-10-14T00:29:30.991 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:29:30.991 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/981386002 ]] 2023-10-14T00:29:30.992 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/981386002 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\9\8\1\3\8\6\0\0\2 ]] 2023-10-14T00:29:30.992 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:29:32.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.518+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:32.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:32.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:32.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:32.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:32.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:32.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.582+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:32.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.582+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:32.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:32.582+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:34.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:34.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:34.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:34.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:34.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:34.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:34.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:34.126+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:34.961 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:34.962+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:29:34.972 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:34.974+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:35.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:35.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:35.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:35.870+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:35.939 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:35.942+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:29:35.954 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:35.954+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:38.031 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:38.034+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:38.041 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:38.042+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:38.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.318+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:38.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:38.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:38.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:38.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:38.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:38.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.370+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:38.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.370+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:38.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:38.370+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:39.406 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:39.406+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:39.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:39.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:39.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:39.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:39.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.642+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:39.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.642+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:39.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.662+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:39.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.950+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:39.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:39.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:39.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:40.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:40.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:40.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:40.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.034+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:40.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.034+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:40.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.041+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:40.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.077+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:29:40.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.077+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:29:40.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.077+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:40.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:40.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:40.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:40.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:40.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:40.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.145+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:40.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.145+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:40.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.149+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:40.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:29:40.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:29:40.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:40.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:40.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:40.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:40.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:40.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:40.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:40.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:40.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.157+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:40.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:40.197+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:41.014 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:29:41.014 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:29:41.020 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:29:41.035 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:29:41.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:41.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:41.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:41.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:41.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:41.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.425+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:41.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.425+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:41.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.433+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:41.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:41.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:42.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:42.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:42.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:42.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:42.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:42.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:42.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:42.041+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:42.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:42.041+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:42.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:42.041+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:42.428 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/981386002 2023-10-14T00:29:42.439 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:29:42.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243382 2023-10-14T00:29:42.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:29:42.440 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/981386002 ]] 2023-10-14T00:29:42.440 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/981386002 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\9\8\1\3\8\6\0\0\2 ]] 2023-10-14T00:29:42.440 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:29:43.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.881+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:43.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:43.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:43.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:43.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:43.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.929+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:43.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.929+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:43.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.933+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:43.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:43.993+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:45.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:45.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:45.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:45.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:45.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:45.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:45.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:45.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:45.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:45.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:45.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:45.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:46.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:46.613+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:46.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:46.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:46.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:46.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:46.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:46.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:46.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:46.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:46.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:46.649+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:47.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:47.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:47.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:47.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:47.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:47.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:47.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:47.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.357+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:47.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.553+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:47.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.561+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:47.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:47.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:47.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:47.791 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:47.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:29:47.793 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:47.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:47.793 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:47.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:47.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:47.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:47.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:47.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:47.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:47.937+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:48.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:48.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:48.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:48.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:48.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:48.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:48.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:48.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:49.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:49.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:49.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:49.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:49.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:49.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:49.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:49.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:49.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:49.581+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:49.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:49.597+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:50.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:50.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:50.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:50.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:50.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.557+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:50.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.557+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:50.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.565+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:50.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:50.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:50.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:50.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:50.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:50.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.681+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:50.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.681+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:50.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.693+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:50.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.945+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:50.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:50.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:50.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:51.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:51.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:51.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.997+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:51.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:50.997+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:51.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.009+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:51.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:51.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:51.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:51.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:51.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.081+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:51.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.081+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:51.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:51.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:51.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:51.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:51.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.169+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:51.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.169+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:51.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:51.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:51.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:51.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:51.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:51.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.437+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:51.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.437+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:51.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:51.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:51.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:51.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:51.485+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:52.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:52.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:52.456 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:29:52.460 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:29:52.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:52.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:52.469 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:29:52.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:52.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:52.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:52.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:52.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:52.489+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:52.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:52.489+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:52.489 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:29:52.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:52.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:53.276 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:53.277+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:53.282 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:53.285+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:53.498 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:53.493+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:53.499 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:53.493+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:53.713 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/981386002 2023-10-14T00:29:53.726 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:29:53.730 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243393 2023-10-14T00:29:53.730 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:29:53.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/981386002 ]] 2023-10-14T00:29:53.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/981386002 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\9\8\1\3\8\6\0\0\2 ]] 2023-10-14T00:29:53.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:29:57.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:57.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:57.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:57.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:57.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:57.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:57.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:57.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:57.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:57.097+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:57.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:57.097+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:57.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:57.113+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:57.602 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:57.601+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:29:57.811 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:29:57.813+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:29:59.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.129+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:29:59.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.129+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:29:59.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.129+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:29:59.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:59.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:59.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:59.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:59.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:59.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.169+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:59.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.169+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:59.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.193+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:59.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:29:59.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:29:59.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:29:59.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:29:59.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:29:59.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.961+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:29:59.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.961+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:29:59.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:29:59.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:29:59.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.021+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.021+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.021+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.021+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352fb00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.021+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352f600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.021+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352f400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.034 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.037+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.035 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.037+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352ee80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.041+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.041+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb332e900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.053+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.053+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3528e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.064 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.065+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.064 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.065+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3528e80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.069+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.067 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.069+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3528f80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.077+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.077+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3529200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.077+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.076 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.077+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3529380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.078 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.081+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.079 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.081+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3528800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.094 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.093+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.094 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.093+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb332fd80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.097+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.097+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3529b00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.109+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.109+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3529e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.109+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.109+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.117+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.117+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.125+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.125+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.145+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.145+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e600 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.150 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.149+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.150 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.149+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.151 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.153+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.151 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:30:00.153+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221ea00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:30:00.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:00.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:00.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.285+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:00.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:00.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:00.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:00.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:00.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:00.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.517+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:00.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.517+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:00.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.517+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:00.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:00.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:00.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:00.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.689+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:00.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.689+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:00.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:00.705+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:01.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.233+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:01.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.761+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:01.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:01.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:01.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:01.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:01.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:01.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:01.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:01.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:01.829+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:02.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.013+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:02.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:02.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:02.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:02.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:02.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:02.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.089+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:02.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.089+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:02.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.097+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:02.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:02.989+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:03.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:03.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:03.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:03.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:03.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:03.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:03.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:03.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:03.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:03.017+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:03.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:03.017+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:03.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:03.037+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:03.769 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:30:03.769 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:30:03.770 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:30:03.805 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:30:05.373 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/981386002 2023-10-14T00:30:05.373 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:30:05.374 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243405 2023-10-14T00:30:05.375 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:30:05.375 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/981386002 ]] 2023-10-14T00:30:05.375 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/981386002 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\9\8\1\3\8\6\0\0\2 ]] 2023-10-14T00:30:05.375 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:30:07.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:07.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:07.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:07.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:07.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:07.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:07.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:07.608+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:08.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:08.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:08.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:08.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:08.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:08.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.492+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:08.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.492+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:08.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.492+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:08.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:08.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:08.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:08.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:08.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.672+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:08.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.672+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:08.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:08.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:09.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:09.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:09.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:09.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:09.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:09.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.920+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:09.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.920+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:09.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:09.920+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:10.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:10.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:10.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:10.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:10.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:10.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.156+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:10.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.156+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:10.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.192+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:10.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:10.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:10.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:10.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:10.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.552+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:10.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.552+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:10.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:10.576+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:11.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:11.644+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:12.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:12.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:12.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:12.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:12.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:15.098 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:15.096+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:15.099 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:15.100+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:15.377 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:30:15.377 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:30:15.377 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:30:15.377 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:30:16.667 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/981386002 2023-10-14T00:30:16.697 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:30:16.699 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243416 2023-10-14T00:30:16.699 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:30:16.700 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/981386002 ]] 2023-10-14T00:30:16.700 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/981386002 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\9\8\1\3\8\6\0\0\2 ]] 2023-10-14T00:30:16.700 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:30:17.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.512+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:17.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:17.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:17.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:17.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:17.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:17.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.536+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:17.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.536+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:17.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.548+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:17.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.576+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:17.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:17.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:17.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:17.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:17.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:17.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.640+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:17.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.640+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:17.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.648+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:17.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:17.980+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:18.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:18.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:18.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:18.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:18.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:18.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:18.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:18.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:18.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:18.740+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:18.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:18.744+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:19.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.492+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:30:19.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.492+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:30:19.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.560+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:19.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:19.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:19.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:19.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:19.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:19.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.572+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:19.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.572+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:19.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:19.620+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:20.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:20.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:20.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:20.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:20.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:20.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:20.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:20.244+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:20.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:20.244+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:20.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:20.248+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:22.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:22.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:22.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:22.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:22.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:22.484+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:24.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:24.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:24.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:24.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:24.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:24.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:24.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:24.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:25.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:25.024+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:25.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:25.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:25.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:25.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:25.466 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:25.460+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:25.466 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:25.460+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:25.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:25.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:25.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:25.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:25.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:25.496+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:26.716 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:30:26.716 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:30:26.716 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:30:26.718 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:30:27.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:27.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:27.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:27.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:27.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:27.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.748+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:27.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.748+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:27.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:27.748+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:28.110 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3223512488 2023-10-14T00:30:28.118 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:30:28.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243428 2023-10-14T00:30:28.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:30:28.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3223512488 ]] 2023-10-14T00:30:28.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3223512488 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\9\8\1\3\8\6\0\0\2 ]] 2023-10-14T00:30:28.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/3223512488 2023-10-14T00:30:29.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.692+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:30:29.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.692+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:30:29.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:29.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:29.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:29.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.704+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.704+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:29.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:29.704+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:30.431 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/3223512488 until 2023-10-14T01:30:29.821511+0000 (3600 sec) 2023-10-14T00:30:30.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.536+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:30.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:30.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:30.547 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:30:30.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:30.554 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/3223512488 2023-10-14T00:30:30.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:30.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:30.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.560+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:30.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.560+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:30.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:30.560+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:31.192 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:30:31.196+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:30:31.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:31.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:31.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:31.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:31.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:31.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.516+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:31.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.516+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:31.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:31.524+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:31.910 INFO:tasks.workunit.client.0.smithi183.stderr:listed 27 entries 2023-10-14T00:30:31.957 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/3223512488 2023-10-14T00:30:31.957 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:30:32.576 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:32.580+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:32.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:32.836+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:32.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:32.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:32.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:32.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:32.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:32.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:32.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:32.884+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:32.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:32.884+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:32.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:32.900+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:33.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.244+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:33.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:33.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:33.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:33.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:33.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:33.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.276+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:33.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.276+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:33.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:33.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:33.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:33.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:33.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:33.828+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:34.016 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:34.019+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:34.016 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:34.019+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:34.225 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:34.227+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:35.008 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:35.011+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:35.011 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:35.011+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:35.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:35.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:35.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:35.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:35.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:35.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.159+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:35.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.159+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:35.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:35.175+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:37.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:37.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:37.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:37.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:37.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:37.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:37.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:37.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:37.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:37.663+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:37.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:37.663+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:37.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:37.679+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:37.963 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:37.967+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:39.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:39.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:39.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:39.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:39.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:39.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.411+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:39.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.411+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:39.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:39.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:39.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:39.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:39.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:39.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:39.791+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:39.808 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:39.811+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:30:39.826 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:39.827+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:30:39.832 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:39.835+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:39.838 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:39.835+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:40.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.415+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:40.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:40.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:40.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:40.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:40.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:40.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.587+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:40.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.587+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:40.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.599+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:40.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:40.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:40.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:40.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:40.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:40.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:40.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:41.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:41.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:41.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:41.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:41.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:41.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:41.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:41.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:41.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:41.927+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:41.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:41.927+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:41.986 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:30:41.991 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:30:41.995 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:30:42.020 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:30:42.031 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:30:42.035+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:30:42.031 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:30:42.035+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:30:42.031 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:30:42.035+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T00:30:42.031 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:30:42.035+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aff790 2023-10-14T00:30:42.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:42.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:42.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:42.491+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:42.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:42.951+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:42.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:42.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:43.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:43.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:43.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:43.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:43.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:43.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:43.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:43.023+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:43.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:43.023+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:43.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:43.031+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:43.652 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3223512488 2023-10-14T00:30:43.656 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:30:43.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243443 2023-10-14T00:30:43.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:30:43.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3223512488 ]] 2023-10-14T00:30:43.659 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3223512488 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\2\3\5\1\2\4\8\8 ]] 2023-10-14T00:30:43.659 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:30:44.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.811+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:30:44.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.815+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:44.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:44.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:44.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:44.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:44.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:44.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.843+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:44.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.843+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:44.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.875+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:44.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:44.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:45.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:45.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:45.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:45.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.999+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:45.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:44.999+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:45.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:45.011+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:47.387 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:47.391+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:47.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:47.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:47.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:47.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:47.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:47.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:47.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:47.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:47.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:47.655+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:47.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:47.655+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:47.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:47.671+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:49.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:49.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:50.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:50.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:50.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:50.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:50.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:50.059+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:50.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:50.067+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:50.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:50.075+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:50.731 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:50.735+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:50.735 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:50.739+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:51.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:51.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:51.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:51.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:51.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:51.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.647+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:51.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.647+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:51.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:51.659+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:52.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:52.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:52.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:52.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:52.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:52.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:52.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:52.567+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:52.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:52.567+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:52.629 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:52.631+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:30:52.645 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:52.647+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:52.653 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:52.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:53.670 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:30:53.671 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:30:53.676 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:30:53.683 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:30:53.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:53.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:53.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:53.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:53.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:53.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.935+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:53.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.935+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:53.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:53.955+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:54.106 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:54.107+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:54.121 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:54.123+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:30:54.254 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:30:54.251+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:30:54.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:54.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:54.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:54.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:54.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:54.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.843+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:54.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.843+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:54.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.843+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:54.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:54.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:55.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:55.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:55.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:55.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:55.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:55.035+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:55.081 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3223512488 2023-10-14T00:30:55.099 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:30:55.101 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243455 2023-10-14T00:30:55.101 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:30:55.101 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3223512488 ]] 2023-10-14T00:30:55.102 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3223512488 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\2\3\5\1\2\4\8\8 ]] 2023-10-14T00:30:55.102 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:30:56.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.611+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:30:56.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:56.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:56.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:56.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:56.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:56.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.703+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:56.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.703+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:30:56.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:56.703+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:30:59.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:59.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:30:59.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:59.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:30:59.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:59.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:30:59.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:59.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:30:59.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:59.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:30:59.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:59.895+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:30:59.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:30:59.907+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:00.000 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.003+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.000 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.003+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2fbff00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.005 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.007+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.005 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.007+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3179000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.011+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.011+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e4c80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.059+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.059+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.059 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.059+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb32e4180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.060 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.059+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb332eb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2d93500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2930a80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.100 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2ffad00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.101 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.103+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3259780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.131+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.131+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.131+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.131+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3259880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.131+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.131+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2d92680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.135+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.131 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.135+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb352eb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.135+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.135+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3212580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.170 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.171+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.171 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.171+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3258500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.193 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.195+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.194 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.195+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.194 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.195+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb352fd00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.194 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.195+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb352ff00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.210 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.211+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.210 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.211+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb352f980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.219 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.219+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.219 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.219+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3493100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.220 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.223+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.221 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:31:00.223+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352f680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:31:00.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.479+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:31:00.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.487+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:31:00.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.491+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:00.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:00.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:00.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:00.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:00.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:00.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.499+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:00.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.499+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:00.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.499+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:00.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.679+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:00.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:00.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:00.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:00.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:00.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.711+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:00.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.711+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:00.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.715+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:00.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.727+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:00.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:00.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:00.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:00.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:00.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:00.775+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:03.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:03.071+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:31:03.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:03.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:03.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:03.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:03.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:03.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:03.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:03.119+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:05.130 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:31:05.136 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:31:05.146 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:31:05.151 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:31:05.386 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:31:05.386+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:31:06.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:06.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:06.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:06.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:06.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:06.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:06.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.142+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:06.211 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3223512488 2023-10-14T00:31:06.211 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:31:06.213 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243466 2023-10-14T00:31:06.213 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:31:06.213 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3223512488 ]] 2023-10-14T00:31:06.214 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3223512488 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\2\3\5\1\2\4\8\8 ]] 2023-10-14T00:31:06.214 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:31:06.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.898+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:06.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:06.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:06.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:06.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:06.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.926+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:06.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.926+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:06.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:06.946+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:07.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:07.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:07.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:07.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:07.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.206+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:07.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.206+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:07.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:07.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:07.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:07.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:07.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.398+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:07.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.398+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:07.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:07.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:07.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:07.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:07.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:07.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:07.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:07.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:07.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.802+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:07.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.802+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:07.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.822+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:08.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:07.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:08.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:08.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:08.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:08.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.038+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:08.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.038+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:08.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:08.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:08.923 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:08.926+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:08.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:08.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:08.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:09.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:09.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:09.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:09.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:09.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.290+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:09.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:09.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:09.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:09.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:09.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:09.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:09.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:10.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.514+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:10.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:10.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:10.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:10.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:10.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:10.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.558+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:10.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.558+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:10.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:10.562+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:12.939 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:12.938+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:31:12.970 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:12.970+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:31:12.977 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:12.978+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:12.985 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:12.986+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:13.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:13.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:13.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.142+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:13.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:13.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:13.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:13.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:13.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:13.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.858+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:13.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.858+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:13.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:13.858+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:16.216 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:31:16.222 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:31:16.231 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:31:16.235 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:31:17.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:17.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:17.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:17.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:17.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:17.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:17.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:17.330+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:17.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:17.330+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:17.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:17.362+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:17.686 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3223512488 2023-10-14T00:31:17.687 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:31:17.687 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243477 2023-10-14T00:31:17.688 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:31:17.688 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3223512488 ]] 2023-10-14T00:31:17.688 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3223512488 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\2\3\5\1\2\4\8\8 ]] 2023-10-14T00:31:17.688 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:31:18.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:18.890+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:19.949 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:19.946+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:19.949 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:19.946+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:19.996 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:19.998+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:20.656 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:20.658+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:20.666 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:20.666+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:22.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.478+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:22.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:22.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:22.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:22.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:22.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:22.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.550+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:22.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.550+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:22.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.550+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:22.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:22.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:22.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:22.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:22.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.658+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:22.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:22.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:22.927 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:22.930+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:23.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:23.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:23.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:23.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:23.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.166+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:23.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.170+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:23.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.194+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:23.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.214+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:31:23.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.230+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:23.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:23.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:23.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:23.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:23.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:23.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.234+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:23.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.234+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:23.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:23.250+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:24.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:24.990+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:24.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:24.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:25.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:25.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.010+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:25.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.010+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:25.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.010+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:25.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.010+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:25.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.010+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:25.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:25.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:25.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:25.306+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:25.673 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:25.674+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:31:25.700 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:25.702+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:26.119 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:26.118+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:26.124 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:26.126+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:26.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:26.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:26.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:26.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:26.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:26.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:26.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:26.722+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:26.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:26.722+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:26.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:26.742+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:27.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.274+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:27.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:27.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:27.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:27.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:27.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.374+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:27.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.374+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:27.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:27.374+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:27.698 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:31:27.698 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:31:27.699 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:31:27.699 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:31:28.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:28.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:28.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:28.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:28.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:28.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:28.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:28.842+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:28.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:28.842+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:28.855 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/3223512488 2023-10-14T00:31:28.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:28.858+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:28.858 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:31:28.875 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243488 2023-10-14T00:31:28.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:31:28.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/3223512488 ]] 2023-10-14T00:31:28.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/3223512488 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\2\3\5\1\2\4\8\8 ]] 2023-10-14T00:31:28.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:31:29.822 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:29.822+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:31:29.833 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:29.834+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:29.833 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:29.834+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:30.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:30.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:30.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:30.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:30.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.022+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:30.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.022+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:30.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.022+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:30.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.130+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:30.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:30.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:30.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:30.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:30.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:30.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.250+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:30.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.250+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:30.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.250+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:30.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:30.367 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:30.370+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:31:30.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:30.379 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:30.382+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:31:30.379 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:30.382+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:30.380 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:30.382+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:30.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:30.382+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:31.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:31.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:32.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:32.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:32.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:32.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:32.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.102+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:32.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.102+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:32.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.126+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:32.442 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:32.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:32.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:32.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:32.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:32.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:32.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.569+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:32.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.569+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:32.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:32.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:32.958 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:32.957+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:34.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:34.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:34.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:34.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:34.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:34.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:34.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.413+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:34.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:34.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:34.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:35.409 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:35.409+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:35.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:35.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:35.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:35.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:35.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:35.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.845+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:35.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.845+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:35.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:35.849+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:38.885 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:31:38.887 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:31:38.896 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:31:38.910 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:31:39.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:39.961+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:39.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:39.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:39.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:39.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:39.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:39.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:39.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:39.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:40.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.001+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:40.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.017+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:40.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/176881915 2023-10-14T00:31:40.343 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:31:40.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243500 2023-10-14T00:31:40.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:31:40.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/176881915 ]] 2023-10-14T00:31:40.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/176881915 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\3\2\2\3\5\1\2\4\8\8 ]] 2023-10-14T00:31:40.344 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/176881915 2023-10-14T00:31:40.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.345+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:40.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:40.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:40.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:40.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:40.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:40.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.413+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:40.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.413+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:40.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.413+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:40.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:40.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:40.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.697+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:40.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:40.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:40.929+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:42.446 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/176881915 until 2023-10-14T01:31:41.695962+0000 (3600 sec) 2023-10-14T00:31:42.517 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:31:42.518 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/176881915 2023-10-14T00:31:42.664 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:42.665+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:43.772 INFO:tasks.workunit.client.0.smithi183.stderr:listed 28 entries 2023-10-14T00:31:43.844 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/176881915 2023-10-14T00:31:43.845 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:31:44.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.177+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:44.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:44.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:44.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:44.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:44.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:44.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.241+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:44.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.241+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:44.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:44.241+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:44.506 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:44.505+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:31:45.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.385+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:45.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:45.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:45.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:45.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:45.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:45.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.417+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:45.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.417+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:45.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.429+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:45.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.433+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:31:45.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.449+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:31:45.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.449+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:45.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:45.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:45.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:45.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:45.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:45.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.477+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:45.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.477+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:45.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:45.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:47.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.037+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:31:47.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:47.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:47.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.073+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:47.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:47.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:47.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:47.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:47.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:47.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:47.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:47.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:47.629+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:48.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.073+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:48.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:48.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:48.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:48.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.861+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:48.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:48.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:48.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:48.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:48.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:48.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.897+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:48.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.901+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:48.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:48.901+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:49.855 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:31:49.853+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:31:49.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.889+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:49.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:49.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:49.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:49.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:49.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:49.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.925+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:49.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.925+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:49.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:49.941+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:51.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:51.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:51.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:51.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:51.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.465+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:51.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.465+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:51.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:51.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:51.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:51.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:51.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:51.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:51.553+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:52.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.149+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:52.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:52.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:52.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:52.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:52.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.197+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:52.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.197+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:52.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:52.225+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:53.862 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:31:53.863 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:31:53.863 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:31:53.874 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:31:54.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:54.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:54.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:54.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:54.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:54.797+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:55.064 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/176881915 2023-10-14T00:31:55.065 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:31:55.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243515 2023-10-14T00:31:55.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:31:55.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/176881915 ]] 2023-10-14T00:31:55.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/176881915 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\7\6\8\8\1\9\1\5 ]] 2023-10-14T00:31:55.067 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:31:55.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:55.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:55.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:55.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:55.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.313+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:55.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:55.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:55.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:55.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:55.461+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:56.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:56.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:56.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:56.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:56.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:56.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:56.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:56.669+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:58.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:58.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:58.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:58.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:58.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:58.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.480+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:58.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.480+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:58.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:31:58.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.696+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:31:58.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.700+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:31:58.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:31:58.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:31:58.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:31:58.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:31:58.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:31:58.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.740+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:31:58.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.740+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:31:58.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:31:58.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:00.003 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.000+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.003 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.000+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352e580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.004 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.000+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.004 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.000+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352e080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.022 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.020+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.022 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.020+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2c51380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.023 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.020+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.024 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.020+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352fd00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.036 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.036+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.036 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.036+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3257e00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.036+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.036+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.037 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.036+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.038 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.036+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.080+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.080+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.080+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.080+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221f100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221ef80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.093 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.094 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221ec80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.094 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb26bab00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.094 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221f380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221f580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.095 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.092+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221f780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.096 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.096+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.097 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.096+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221f980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.097 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.096+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.097 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.096+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221fb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.116+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.118 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.116+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.116+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.116+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221fd80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.116+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221ff80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.119 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:32:00.116+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbe180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:32:00.358 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:00.356+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:32:00.378 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:00.376+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:32:00.392 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:00.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:00.392 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:00.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:01.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.464+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:32:01.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.464+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:32:01.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.492+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:01.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:01.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:01.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:01.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:01.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:01.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.520+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:01.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.520+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:01.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.520+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:01.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.912+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:01.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:01.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:01.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:01.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:01.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:01.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.968+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:01.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.968+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:01.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:01.988+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:02.195 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:02.188+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:02.195 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:02.192+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:02.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:02.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:02.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:02.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:03.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:02.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:03.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:02.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:03.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:02.996+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:03.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:02.996+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:03.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:02.996+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:03.402 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:03.400+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:03.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:03.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:03.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:03.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:03.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:03.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.740+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:03.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.740+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:03.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:03.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:04.546 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:04.544+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:32:04.546 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:04.544+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:04.558 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:04.556+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:04.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.684+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:04.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:04.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:04.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:04.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:04.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:04.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.732+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:04.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.732+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:04.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:04.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:04.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:04.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:04.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:04.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.880+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:32:04.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.880+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:32:04.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.892+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:04.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:04.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:04.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:04.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:04.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:04.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:05.072 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:32:05.073 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:32:05.091 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:32:05.094 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:32:06.657 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:06.656+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:06.705 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:06.704+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:32:06.732 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:06.732+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:06.774 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/176881915 2023-10-14T00:32:06.774 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:32:06.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243526 2023-10-14T00:32:06.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:32:06.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/176881915 ]] 2023-10-14T00:32:06.780 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/176881915 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\7\6\8\8\1\9\1\5 ]] 2023-10-14T00:32:06.781 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:32:08.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:08.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.616+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:08.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:08.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:08.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:08.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.664+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:08.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.664+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:08.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:08.668+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:08.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:08.868+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:08.944 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:08.892+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:09.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:09.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:09.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:09.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:09.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:09.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.104+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:09.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.104+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:09.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.116+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:09.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:09.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:09.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:09.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.220+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:32:09.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:09.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:09.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:09.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:09.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:12.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.644+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:12.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:12.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:12.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:12.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:12.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:12.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.700+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:12.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.700+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:12.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:12.712+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:13.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:13.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:13.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:13.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:13.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:13.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.856+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:13.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.856+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:13.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.856+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:13.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:13.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:14.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.656+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:14.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:14.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:14.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:14.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:14.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:14.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.676+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:14.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.676+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:14.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:14.720+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:16.782 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:32:16.782 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:32:16.782 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:32:16.785 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:32:18.143 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/176881915 2023-10-14T00:32:18.143 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:32:18.150 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243538 2023-10-14T00:32:18.150 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:32:18.150 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/176881915 ]] 2023-10-14T00:32:18.151 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/176881915 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\7\6\8\8\1\9\1\5 ]] 2023-10-14T00:32:18.151 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:32:18.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:18.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:18.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:18.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:18.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:18.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:18.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:18.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:18.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:18.592+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:18.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:18.592+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:18.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:18.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:20.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.080+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:20.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:20.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:20.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:20.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:20.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.136+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:20.175 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:20.176+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:32:20.246 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:20.248+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:20.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:20.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:20.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:20.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:20.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:20.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:20.588+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:21.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:21.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:21.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:21.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:21.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.160+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:21.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.160+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:21.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.180+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:21.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.588+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:21.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:21.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:21.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:21.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:21.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:21.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.664+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:21.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.664+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:21.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:21.704+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:22.987 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:22.988+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:24.984 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:24.983+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:24.985 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:24.987+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:27.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.251+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:27.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:27.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:27.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:27.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:27.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:27.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.299+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:27.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.299+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:27.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:27.299+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:28.152 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:32:28.162 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:32:28.162 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:32:28.172 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:32:29.561 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/176881915 2023-10-14T00:32:29.562 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:32:29.563 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243549 2023-10-14T00:32:29.563 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:32:29.563 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/176881915 ]] 2023-10-14T00:32:29.563 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/176881915 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\7\6\8\8\1\9\1\5 ]] 2023-10-14T00:32:29.564 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:32:30.210 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:30.211+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:32.195 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:32.195+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:32.411 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:32.411+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:33.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.723+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:33.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:33.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:33.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:33.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:33.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:33.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.783+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:33.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.783+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:33.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:33.811+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:34.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:34.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:34.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:34.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:34.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:34.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.863+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:32:34.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:34.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:34.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:34.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:34.998 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:34.999+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:36.150 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:36.151+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:38.050 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:32:38.051+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:32:38.051 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:32:38.051+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:32:38.051 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:32:38.051+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T00:32:38.051 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:32:38.051+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aff790 2023-10-14T00:32:38.051 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:32:38.051+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69b73ba0 2023-10-14T00:32:38.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:38.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:38.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:38.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:38.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:38.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.323+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:38.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.323+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:38.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:38.323+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:39.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:39.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:39.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:39.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:39.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:39.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:39.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:39.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:39.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:39.027+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:39.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:39.027+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:39.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:39.031+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:39.565 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:32:39.569 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:32:39.573 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:32:39.599 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:32:40.853 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/176881915 2023-10-14T00:32:40.854 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:32:40.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243560 2023-10-14T00:32:40.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:32:40.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/176881915 ]] 2023-10-14T00:32:40.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/176881915 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\7\6\8\8\1\9\1\5 ]] 2023-10-14T00:32:40.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:32:41.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:41.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:41.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:41.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:41.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.571+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:41.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.571+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:41.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.571+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:41.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:41.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:41.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:41.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:45.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:45.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:45.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:45.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:45.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:45.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:45.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:45.511+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:45.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:45.511+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:46.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:46.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:46.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:46.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:46.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.327+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:46.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.327+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:46.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.347+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:46.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:46.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:46.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:46.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:46.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:46.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.739+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:46.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.739+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:46.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.739+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:46.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.907+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:46.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:46.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:46.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:46.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:46.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:46.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:46.939+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:47.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:47.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:47.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:47.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:48.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:48.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:48.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:48.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.051+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:48.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.051+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:48.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.067+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:48.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.523+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:32:48.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.523+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:32:48.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.539+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:48.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:48.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:32:48.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:48.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:48.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:48.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.575+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:48.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.575+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:48.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:48.619+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:49.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:49.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:49.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:49.515+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:49.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:49.515+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:49.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:49.515+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:49.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:49.515+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:49.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:49.535+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:50.871 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:32:50.871 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:32:50.872 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:32:50.892 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:32:50.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:50.910+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:32:50.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:50.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:32:50.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:50.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:32:50.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:50.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:32:51.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:51.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:32:51.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:51.006+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:32:51.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:51.006+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:32:51.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:32:51.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:32:52.437 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2403304423 2023-10-14T00:32:52.438 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:32:52.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243572 2023-10-14T00:32:52.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:32:52.439 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2403304423 ]] 2023-10-14T00:32:52.440 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2403304423 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\1\7\6\8\8\1\9\1\5 ]] 2023-10-14T00:32:52.440 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2403304423 2023-10-14T00:32:54.447 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2403304423 until 2023-10-14T01:32:53.770994+0000 (3600 sec) 2023-10-14T00:32:54.475 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:32:54.475 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2403304423 2023-10-14T00:32:55.477 INFO:tasks.workunit.client.0.smithi183.stderr:listed 29 entries 2023-10-14T00:32:55.512 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2403304423 2023-10-14T00:32:55.513 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:32:57.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:57.402+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:32:57.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:57.402+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:57.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:57.402+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:57.979 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:57.978+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:32:59.286 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:59.286+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:32:59.300 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:59.298+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:32:59.318 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:59.318+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:32:59.319 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:32:59.318+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.002+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.002+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3256080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.052 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.054+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.104 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.054+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb221ea80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.106 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352f000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.107 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.070+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.086+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.086+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.086+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.108 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.086+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.109 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.109 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbef00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.109 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.109 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbed80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.109 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbeb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb221ea80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352e780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3259a80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.094+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb352f000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.098+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.098+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.098+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.098+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.125 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.126+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.126+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe800 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.126+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.126 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.126+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbef00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.126+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.127 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:33:00.126+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbed80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:33:00.503 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:00.502+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:01.714 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:01.714+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:33:01.740 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:01.738+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:02.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:02.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:02.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:02.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:02.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:02.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:02.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:02.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:02.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:02.402+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:02.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:02.402+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:02.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:02.402+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:05.519 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:33:05.519 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:33:05.527 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:33:05.543 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:33:06.855 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2403304423 2023-10-14T00:33:06.856 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:33:06.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243586 2023-10-14T00:33:06.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:33:06.857 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2403304423 ]] 2023-10-14T00:33:06.858 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2403304423 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\0\3\3\0\4\4\2\3 ]] 2023-10-14T00:33:06.858 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:33:07.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:07.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:07.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:07.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:07.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:07.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:07.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:07.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:07.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:07.134+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:08.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:08.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:08.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:08.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:08.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:08.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.402+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:08.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.402+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:08.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:08.410+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:09.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.162+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:09.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:09.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:09.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:09.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:09.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.254+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:09.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.254+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:09.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:09.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:09.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.762+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:09.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:09.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:09.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:09.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.778+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:09.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.778+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:09.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.778+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:09.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:09.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:09.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:09.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:09.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:09.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.986+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:09.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.994+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:09.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:09.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:12.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:12.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:12.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:12.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:12.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:12.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:12.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:12.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:12.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:12.514+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:12.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:12.514+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:12.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:12.514+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:13.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:13.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:13.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:13.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:13.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:13.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:13.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:13.406+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:13.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:13.438+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:16.869 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:33:16.872 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:33:16.872 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:33:16.877 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:33:16.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:16.974+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:17.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:17.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:17.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:17.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:17.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:17.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:17.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:17.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:17.046+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:18.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.142+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:18.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:18.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:18.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:18.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:18.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.158+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:18.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.158+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:18.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:18.158+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:18.268 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2403304423 2023-10-14T00:33:18.270 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:33:18.272 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243598 2023-10-14T00:33:18.272 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:33:18.272 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2403304423 ]] 2023-10-14T00:33:18.272 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2403304423 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\0\3\3\0\4\4\2\3 ]] 2023-10-14T00:33:18.272 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:33:22.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:22.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:22.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:22.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:22.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:22.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:22.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:22.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:23.353 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:23.349+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:33:23.353 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:23.349+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:33:23.360 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:23.357+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:33:23.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:23.357+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:24.048 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:24.045+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:33:24.061 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:24.061+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:24.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:24.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:24.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:24.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:24.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:24.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:24.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:24.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:24.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:24.221+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:24.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:24.221+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:24.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:24.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:25.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:25.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:25.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:25.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:25.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:25.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.265+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:25.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.265+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:25.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:25.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.765+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:25.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:25.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:25.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:25.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:25.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:25.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.821+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:25.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.821+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:25.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:25.821+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:26.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:26.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:26.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:26.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:26.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:26.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:26.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:26.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:26.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:26.381+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:28.059 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:28.057+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:28.281 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:33:28.282 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:33:28.283 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:33:28.283 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:33:28.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.405+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:28.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:28.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:28.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:28.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:28.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:28.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.461+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:28.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.461+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:28.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:28.465+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:29.459 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2403304423 2023-10-14T00:33:29.460 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:33:29.460 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243609 2023-10-14T00:33:29.460 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:33:29.460 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2403304423 ]] 2023-10-14T00:33:29.461 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2403304423 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\0\3\3\0\4\4\2\3 ]] 2023-10-14T00:33:29.461 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:33:33.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.289+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:33.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:33.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:33.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:33.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:33.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.305+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:33.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.305+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:33.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:33.305+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:35.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:35.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:35.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:35.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:35.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:35.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:35.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:35.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:35.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:35.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:35.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:35.505+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:36.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:36.321+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:33:39.462 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:33:39.463 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:33:39.464 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:33:39.478 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:33:40.929 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2403304423 2023-10-14T00:33:40.958 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:33:40.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243620 2023-10-14T00:33:40.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:33:40.965 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2403304423 ]] 2023-10-14T00:33:40.966 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2403304423 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\0\3\3\0\4\4\2\3 ]] 2023-10-14T00:33:40.966 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:33:41.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.441+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:41.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:41.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:41.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:41.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.537+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:41.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.537+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:41.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:41.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.585+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:41.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.585+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:41.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:41.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:41.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:41.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:41.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.661+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:41.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.661+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:41.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:41.681+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:43.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:43.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:43.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:43.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:43.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:43.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.905+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:43.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.905+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:43.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:43.905+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:45.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.633+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:45.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:45.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:45.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:45.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:45.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:45.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:45.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.825+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:45.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:45.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:45.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:45.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.837+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:45.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.837+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:45.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:45.837+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:46.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:46.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:46.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:46.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:46.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.605+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:46.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.605+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:46.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.605+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:46.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.805+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:46.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:46.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:46.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:46.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:46.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.857+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:46.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.857+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:46.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:46.905+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:47.227 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:47.225+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:33:47.269 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:47.269+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:47.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:47.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:47.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:47.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:47.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.557+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:47.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.557+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:47.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.557+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:47.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.805+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:47.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:47.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:47.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:47.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:47.945+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:33:47.963 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:47.961+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:48.059 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:48.057+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:33:48.062 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:48.061+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:33:48.071 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:48.069+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:48.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.569+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:48.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:48.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:48.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:48.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:48.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:48.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.665+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:48.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.665+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:48.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:48.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:48.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:48.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:48.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.869+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:48.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:48.877+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:49.673 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:49.669+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:50.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:50.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:50.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:50.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:50.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:50.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:50.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:50.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:50.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:50.145+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:50.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:50.145+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:50.970 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:33:50.970 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:33:50.992 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:33:51.008 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:33:51.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.412+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:51.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:51.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:51.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:51.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:51.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.472+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:51.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:51.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:51.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:51.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:51.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:51.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.800+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:51.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:51.816+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:52.521 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2403304423 2023-10-14T00:33:52.522 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:33:52.523 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243632 2023-10-14T00:33:52.523 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:33:52.523 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2403304423 ]] 2023-10-14T00:33:52.523 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2403304423 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\0\3\3\0\4\4\2\3 ]] 2023-10-14T00:33:52.523 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:33:52.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:52.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:52.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:52.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:52.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:52.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:52.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:52.756+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:52.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:52.756+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:52.992 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:52.988+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:33:52.997 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:33:52.996+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:33:53.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:53.244+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:33:53.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:53.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:53.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:53.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:53.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:53.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:53.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:53.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:33:53.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:53.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:54.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:54.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:54.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:54.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:54.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:54.000+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:54.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:54.000+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:54.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:54.024+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:56.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.132+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:56.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.132+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:56.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:56.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:56.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.136+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:56.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.136+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:56.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:56.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:56.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:56.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:56.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:56.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:56.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.240+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:33:56.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:33:56.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:33:56.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:33:56.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:33:56.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.264+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:33:56.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.264+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:33:56.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:33:56.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:02.537 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:34:02.541 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:34:02.541 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:34:02.545 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:34:03.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:03.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:03.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:03.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:03.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:03.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.868+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:03.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.868+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:03.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:03.876+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:04.083 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2795815641 2023-10-14T00:34:04.090 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:34:04.098 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243644 2023-10-14T00:34:04.098 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:34:04.098 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2795815641 ]] 2023-10-14T00:34:04.098 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2795815641 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\4\0\3\3\0\4\4\2\3 ]] 2023-10-14T00:34:04.099 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2795815641 2023-10-14T00:34:04.432 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:04.428+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:34:04.436 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:04.432+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:04.436 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:04.432+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:05.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:05.548+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:05.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:05.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:05.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:05.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:05.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:05.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:05.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:05.568+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:06.485 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2795815641 until 2023-10-14T01:34:05.801030+0000 (3600 sec) 2023-10-14T00:34:06.543 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:34:06.543 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2795815641 2023-10-14T00:34:06.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:06.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:06.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:06.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:06.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:06.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:06.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:06.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:06.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:06.852+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:06.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:06.852+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:06.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:06.880+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:07.499 INFO:tasks.workunit.client.0.smithi183.stderr:listed 30 entries 2023-10-14T00:34:07.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:07.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:07.554 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2795815641 2023-10-14T00:34:07.554 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:34:07.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:07.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:07.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:07.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:07.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:07.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:07.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:08.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.544+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:08.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:08.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:08.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:08.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:08.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.600+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:08.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.600+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:08.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:08.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:09.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:09.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:09.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:09.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.172+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:09.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.188+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:09.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.196+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:09.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.368+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:09.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:09.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:09.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:09.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:09.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.420+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:09.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.420+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:09.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:09.420+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:10.020 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:10.016+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:10.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.036+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:34:10.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.036+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:34:10.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.044+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:10.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:10.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:10.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:10.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:10.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:10.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.096+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:10.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.096+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:10.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:10.116+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:11.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:11.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:11.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:11.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:11.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:11.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:12.196 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:12.192+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:34:12.213 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:12.208+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:12.217 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:12.216+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:12.366 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:12.364+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:12.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:12.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:12.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:12.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:12.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:12.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:12.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:12.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:12.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:12.652+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:12.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:12.656+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:13.092 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:13.088+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:34:13.092 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:13.088+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:34:13.093 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:13.088+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:13.093 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:13.088+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:13.621 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:13.620+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:14.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.612+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:14.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:14.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:14.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:14.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:14.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:14.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.684+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:14.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.684+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:14.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:14.684+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:14.834 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:14.836+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:34:14.836 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:14.836+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:14.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:14.872+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:16.328 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:16.328+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:34:16.378 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:16.380+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:16.378 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:16.380+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:16.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:16.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:16.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:16.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:16.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:16.988+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:17.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.152+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:34:17.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:17.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:17.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:17.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:17.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:17.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:17.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.208+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:17.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.212+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:17.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.220+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:34:17.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.224+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:17.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:17.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:17.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:17.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:17.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:17.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.300+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:17.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.300+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:17.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:17.332+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:17.558 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:34:17.563 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:34:17.576 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:34:17.593 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:34:18.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.432+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:34:18.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.432+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:34:18.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.484+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:18.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:18.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:18.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:18.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:18.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:18.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.508+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:18.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.508+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:18.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:18.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:18.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:18.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:18.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.784+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:18.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.784+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:18.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.784+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:18.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:18.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:19.027 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2795815641 2023-10-14T00:34:19.028 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:34:19.028 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243659 2023-10-14T00:34:19.028 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:34:19.029 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2795815641 ]] 2023-10-14T00:34:19.029 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2795815641 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\7\9\5\8\1\5\6\4\1 ]] 2023-10-14T00:34:19.029 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:34:19.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:19.948+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:22.980 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:22.979+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:34:23.005 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:23.007+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:23.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:23.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:23.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:23.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:23.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.063+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:23.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.111+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:23.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:23.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:23.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:23.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:23.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:23.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:23.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:24.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:24.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:24.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:24.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:24.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:24.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.155+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:24.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.155+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:24.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.155+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:24.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:24.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:24.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:24.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:24.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:24.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:25.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:25.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:25.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:25.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:25.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:25.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:25.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:25.535+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:25.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:25.535+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:25.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:25.543+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:25.855 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:25.855+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:27.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:27.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:27.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:27.091+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:28.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:28.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:28.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:28.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:28.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.335+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:28.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.335+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:28.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.335+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:28.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:28.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:28.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:28.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:28.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:28.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:28.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.851+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:28.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:28.851+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:29.049 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:34:29.053 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:34:29.061 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:34:29.073 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:34:29.823 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:29.823+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:29.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:29.871+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:30.596 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2795815641 2023-10-14T00:34:30.601 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:34:30.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243670 2023-10-14T00:34:30.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:34:30.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2795815641 ]] 2023-10-14T00:34:30.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2795815641 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\7\9\5\8\1\5\6\4\1 ]] 2023-10-14T00:34:30.604 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:34:31.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:31.795+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:31.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:31.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:31.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:31.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:31.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:31.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:31.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:31.831+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:31.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:31.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:32.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:32.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:32.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:32.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:32.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.059+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:32.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.059+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:32.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.071+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:32.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.087+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:32.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:32.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:32.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:32.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:32.139+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:33.386 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:33.387+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:33.706 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:33.707+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:34.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:34.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:34.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:34.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:34.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:34.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:34.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:34.951+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:35.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:35.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:35.592 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:35.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:34:35.611 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:35.611+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:35.627 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:35.627+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:35.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:35.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:35.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:35.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:35.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:35.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.787+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:35.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.787+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:35.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.795+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:35.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:35.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:35.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:35.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:35.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.899+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:35.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.899+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:35.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:35.919+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:37.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:37.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:37.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:37.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:37.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:37.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.131+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:37.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.131+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:37.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.135+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:37.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:37.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:37.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:37.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:37.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.235+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:37.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:37.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:37.611 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:37.611+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:34:37.620 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:37.619+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:37.657 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:37.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:38.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:38.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:38.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:38.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.599+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:38.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.831+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:34:38.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.839+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:38.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:38.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:38.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:38.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:38.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:38.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.895+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:38.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.895+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:38.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:38.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:40.604 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:34:40.607 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:34:40.610 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:34:40.633 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:34:40.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:40.943+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:40.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:40.943+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:40.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:40.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:41.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:41.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:41.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:41.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.003+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:41.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.003+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:41.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.003+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:41.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.139+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:41.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:41.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:41.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:41.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:41.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:41.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.151+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:41.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.151+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:41.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:41.187+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:41.858 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2795815641 2023-10-14T00:34:41.858 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:34:41.858 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243681 2023-10-14T00:34:41.859 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:34:41.859 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2795815641 ]] 2023-10-14T00:34:41.859 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2795815641 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\7\9\5\8\1\5\6\4\1 ]] 2023-10-14T00:34:41.859 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:34:43.087 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:43.083+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:34:43.096 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:43.095+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:44.495 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:44.495+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:34:49.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:49.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:49.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:49.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:49.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:49.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:49.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:49.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:49.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:49.703+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:49.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:49.703+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:49.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:49.739+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:50.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:50.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:50.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:50.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:50.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:50.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.699+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:50.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.699+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:50.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:50.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:50.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:50.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:50.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:50.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.799+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:50.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.799+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:50.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:50.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:50.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:51.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:51.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:51.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:51.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:51.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:51.031+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:51.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:51.031+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:51.446 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:34:51.447+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:34:51.861 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:34:51.877 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:34:51.877 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:34:51.880 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:34:53.508 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2795815641 2023-10-14T00:34:53.518 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:34:53.520 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243693 2023-10-14T00:34:53.520 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:34:53.521 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2795815641 ]] 2023-10-14T00:34:53.521 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2795815641 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\7\9\5\8\1\5\6\4\1 ]] 2023-10-14T00:34:53.521 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:34:54.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.023+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:54.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:54.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:54.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:54.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:54.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.111+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:54.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.111+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:54.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.115+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:54.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:54.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:54.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:54.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.531+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:54.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:54.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:54.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:54.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:54.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.587+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:54.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.775+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:34:54.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:34:54.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:54.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:54.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:54.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:54.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.831+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:54.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.831+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:54.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:54.831+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:55.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:55.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:55.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:55.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.199+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:55.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:55.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:55.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:55.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:55.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.807+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:55.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.807+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:55.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:55.835+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:56.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:56.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:56.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:56.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:56.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:56.339+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:34:58.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:58.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:34:58.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:58.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:34:58.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:58.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:34:58.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:58.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:34:58.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:58.122+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:34:58.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:58.122+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:34:58.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:34:58.134+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:00.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.030+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.038+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.040 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.038+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb31efa00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.038+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.038+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352f980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.038+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.038+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.042+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3256180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.043 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.042+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb352f380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:00.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.058+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.061 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.058+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb3256080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.070+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.070+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbea80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.082+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.082+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.082 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.082+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbf380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.083 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.082+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbf200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:00.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:00.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:00.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.086+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.086+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.086+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbf900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.086+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.088 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.086+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbf780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.091 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.090+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.091 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.090+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbf580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.110+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.110+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb221e680 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.122 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.122+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.123 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.122+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbfb80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.128 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.126+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.129 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.126+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbfd80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.129 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.126+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.129 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.126+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.129 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.126+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb2cbff80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.130 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.126+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb37e2180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.130+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.132 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.130+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb37e2380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.136 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.134+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.136 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.134+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb37e2580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.142+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.142+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb37e2780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.149 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.146+0000 7f660d1ed640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.150 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:35:00.146+0000 7f660d1ed640 -1 librbd::ImageState: 0x55ccb37e2980 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:35:00.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:00.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:00.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.838+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:00.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:00.858+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:03.536 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:35:03.544 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:35:03.546 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:35:03.555 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:35:04.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.118+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:04.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:04.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:04.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:04.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:04.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:04.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.210+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:04.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.210+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:04.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:04.234+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:04.612 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2795815641 2023-10-14T00:35:04.612 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:35:04.621 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243704 2023-10-14T00:35:04.622 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:35:04.622 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2795815641 ]] 2023-10-14T00:35:04.622 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2795815641 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\7\9\5\8\1\5\6\4\1 ]] 2023-10-14T00:35:04.622 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:35:05.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:05.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:05.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:05.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:05.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:05.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.626+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:05.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.626+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:05.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:05.626+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:06.037 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:06.038+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:06.073 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:06.066+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:09.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:09.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:09.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:09.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:09.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:09.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.438+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:09.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.438+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:09.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:09.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:09.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:09.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.598+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:09.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.598+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:09.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:09.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:09.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:09.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:09.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.842+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:09.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:09.854+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:10.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.086+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:10.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:10.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:10.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:10.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.126+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:10.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.126+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:10.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:10.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:10.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:10.138+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:10.344 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:10.342+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:10.408 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:10.406+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:11.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:11.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:11.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:11.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:11.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:11.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:11.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:11.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:11.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:11.154+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:13.019 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:13.018+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:13.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.154+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:13.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:13.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:13.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:13.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:13.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:13.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.190+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:13.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.190+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:13.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:13.210+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:13.502 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:13.502+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:13.519 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:13.518+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:13.915 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:13.914+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:13.945 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:35:13.942+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:35:13.945 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:35:13.942+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:35:13.946 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:35:13.942+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T00:35:13.946 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:35:13.942+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aff790 2023-10-14T00:35:14.624 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:35:14.624 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:35:14.625 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:35:14.625 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:35:14.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.722+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:14.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.734+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:14.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.742+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:14.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:14.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:14.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:14.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:14.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.862+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:14.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.866+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:14.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.866+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:14.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.866+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:14.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:14.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:14.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:14.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:14.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:14.914+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:15.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:15.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:15.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:15.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:15.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.150+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:15.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.150+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:15.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.150+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:15.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.162+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:15.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.162+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:15.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.162+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:15.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:15.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:15.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:15.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:15.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:15.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.174+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:15.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.190+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:15.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.190+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:15.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.402+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:15.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:15.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:15.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:15.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:15.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:15.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.430+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:15.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.430+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:15.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:15.450+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:15.927 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4117264617 2023-10-14T00:35:15.934 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:35:15.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243715 2023-10-14T00:35:15.935 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:35:15.936 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4117264617 ]] 2023-10-14T00:35:15.936 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4117264617 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\7\9\5\8\1\5\6\4\1 ]] 2023-10-14T00:35:15.936 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/4117264617 2023-10-14T00:35:18.529 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/4117264617 until 2023-10-14T01:35:17.862921+0000 (3600 sec) 2023-10-14T00:35:18.593 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:35:18.599 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/4117264617 2023-10-14T00:35:18.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:18.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:18.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:18.986+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:19.692 INFO:tasks.workunit.client.0.smithi183.stderr:listed 31 entries 2023-10-14T00:35:19.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/4117264617 2023-10-14T00:35:19.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:35:19.895 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:19.894+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:35:19.925 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:19.918+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:19.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:19.942+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:20.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:20.942+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:20.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:20.942+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:20.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:20.966+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:20.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:20.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:20.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:20.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:21.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:21.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:21.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:21.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.014+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:21.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.014+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:21.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.026+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:21.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:21.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:21.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:22.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:22.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:22.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:22.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.038+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:22.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.038+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:22.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.038+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:22.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.778+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:22.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:22.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:22.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:22.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:22.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.870+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:22.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.870+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:22.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:22.878+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:24.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:24.462+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:26.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.690+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:26.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:26.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:26.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:26.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:26.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:26.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.742+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:26.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.742+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:26.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:26.750+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:28.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:28.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:28.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:28.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:28.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.218+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:28.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.218+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:28.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.218+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:28.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.218+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:28.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.218+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:28.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.226+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:28.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:28.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:28.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:28.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:28.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.246+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:28.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.246+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:28.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:28.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:28.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:28.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:28.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.638+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:28.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.650+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:28.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.650+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:28.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.650+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:28.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:28.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:28.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:28.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:28.694+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:29.760 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:35:29.760 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:35:29.761 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:35:29.783 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:35:30.067 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:30.066+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:30.082 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:30.082+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:30.409 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:30.406+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:31.193 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4117264617 2023-10-14T00:35:31.194 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:35:31.195 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243731 2023-10-14T00:35:31.195 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:35:31.196 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4117264617 ]] 2023-10-14T00:35:31.196 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4117264617 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\1\7\2\6\4\6\1\7 ]] 2023-10-14T00:35:31.196 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:35:31.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.546+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:31.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:31.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:31.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:31.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:31.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:31.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.566+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:31.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.566+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:31.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:31.598+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:34.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:34.929+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:34.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:34.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:34.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:34.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:34.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:34.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:34.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:34.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:34.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:34.953+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:34.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:34.957+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:36.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.105+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:36.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.105+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:36.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:36.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:36.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:36.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:36.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:36.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.145+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:36.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.145+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:36.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.173+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:36.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.369+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:36.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:36.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:36.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:36.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:36.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.417+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:36.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.429+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:36.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.793+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:36.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:36.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:36.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:36.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:36.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:36.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:36.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.833+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:36.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:36.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:36.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:37.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:37.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:37.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:37.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:37.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.025+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:37.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.025+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:37.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.025+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:37.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.961+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:37.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.961+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:35:37.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.985+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:37.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:37.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:38.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:38.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:38.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:38.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:38.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:38.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:38.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:38.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:38.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:38.029+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:38.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:38.029+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:38.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:38.037+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:40.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:40.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:40.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:40.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:40.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:40.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:40.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:40.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:40.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:40.281+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:40.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:40.281+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:40.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:40.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:41.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:41.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:41.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:41.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:41.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:41.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:41.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:41.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:41.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:41.185+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:41.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:41.185+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:41.201 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:35:41.201 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:35:41.216 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:35:41.217 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:35:41.854 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:41.853+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:42.373 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4117264617 2023-10-14T00:35:42.374 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:35:42.374 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243742 2023-10-14T00:35:42.374 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:35:42.375 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4117264617 ]] 2023-10-14T00:35:42.375 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4117264617 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\1\7\2\6\4\6\1\7 ]] 2023-10-14T00:35:42.375 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:35:42.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:42.893+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:42.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:42.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:42.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:42.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:42.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:42.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:42.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:42.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:42.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:42.953+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:42.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:42.953+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:43.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.813+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:43.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:43.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:43.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:43.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:43.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:43.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:43.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:43.825+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:44.468 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:44.465+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:45.393 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:45.389+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:45.394 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:45.393+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:45.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.617+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:45.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:45.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:45.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:45.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:45.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.697+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:45.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.697+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:45.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:45.717+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:46.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.541+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:46.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:46.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:46.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:46.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:46.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.569+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:46.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.569+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:46.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:46.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.773+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:46.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:46.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:46.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:46.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:46.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:46.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.917+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:46.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.917+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:46.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:46.921+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:47.895 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:47.893+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:48.115 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:48.113+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:48.772 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:48.769+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:49.103 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:49.101+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:49.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:49.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:49.201 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:49.201+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:50.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:50.285+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:50.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:50.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:50.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:50.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:50.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:50.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:50.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:50.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:50.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:50.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:50.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:50.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:51.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.333+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:51.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:51.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:51.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:51.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:51.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.369+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:51.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.369+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:51.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:51.369+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:52.390 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:35:52.391 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:35:52.396 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:35:52.409 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:35:53.178 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:53.177+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:35:53.193 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:53.189+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:53.193 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:53.189+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:53.796 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4117264617 2023-10-14T00:35:53.796 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:35:53.813 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243753 2023-10-14T00:35:53.814 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:35:53.814 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4117264617 ]] 2023-10-14T00:35:53.814 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4117264617 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\1\7\2\6\4\6\1\7 ]] 2023-10-14T00:35:53.814 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:35:53.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.949+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:35:53.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.961+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:53.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:53.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:53.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:54.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:54.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:54.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.997+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:54.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:53.997+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:54.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.005+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:54.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.453+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:54.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:54.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:54.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:54.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:54.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:54.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:54.501+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:54.926 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:54.925+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:35:54.926 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:54.925+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:55.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:55.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:35:55.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:55.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:55.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:55.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:55.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:55.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:55.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:55.573+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:55.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:55.573+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:56.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:56.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:35:56.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:56.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:35:56.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:56.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:35:56.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:56.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:35:56.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:56.449+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:35:56.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:56.449+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:35:56.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:56.465+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:35:59.241 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:59.237+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:35:59.249 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:35:59.249+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:35:59.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:59.985+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:35:59.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:35:59.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:00.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:00.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:00.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:00.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.025+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:00.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.025+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:00.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.025+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:00.098 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.097+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.097+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3178100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.110 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.111 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbed80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e2f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e2580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.112 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e2c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.113 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.115 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.109+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.113+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.116 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.113+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.145+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.145 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.145+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.151 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.149+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.151 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.149+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb3178100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.159 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.157+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.160 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.157+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbed80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.172 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.169+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.172 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.169+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e2f00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.173 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.173+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.173 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.173+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb2cbe500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.180 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.177+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.180 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.177+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.180 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.177+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.181 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.177+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e2580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.181 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.177+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e2c00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.181 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.177+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3380 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.186 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.185+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.186 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.185+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3200 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.200 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.197+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.200 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.197+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.221 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.221+0000 7f660d9ee640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.222 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:00.221+0000 7f660d9ee640 -1 librbd::ImageState: 0x55ccb37e3780 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:00.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:00.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:00.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:00.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:00.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.529+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:00.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:00.529+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:01.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.501+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:01.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:01.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:01.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:01.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:01.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.529+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:01.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.529+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:01.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:01.537+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:03.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:03.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:03.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:03.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.169+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:03.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.169+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:03.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.181+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:03.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.805+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:03.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:03.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:03.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:03.844 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:36:03.850 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:36:03.852 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:36:03.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:03.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:03.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.849+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:03.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.849+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:03.857 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:36:03.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:03.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:04.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:04.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:04.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:04.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:04.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:04.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:04.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:04.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:04.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:04.817+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:04.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:04.845+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:05.162 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4117264617 2023-10-14T00:36:05.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:05.174 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:36:05.176 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243765 2023-10-14T00:36:05.176 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:36:05.176 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4117264617 ]] 2023-10-14T00:36:05.177 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4117264617 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\1\7\2\6\4\6\1\7 ]] 2023-10-14T00:36:05.177 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:36:05.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:05.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:05.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:05.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:05.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.237+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:05.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.237+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:05.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.237+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:05.326 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:05.325+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:05.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:05.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:05.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:05.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:05.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.893+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:05.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.893+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:05.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:05.909+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:09.019 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:09.017+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:11.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:11.336+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:11.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:11.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:11.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:11.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:11.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:11.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:11.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:11.368+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:13.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:13.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:13.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:13.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:13.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:13.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:13.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:13.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:13.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:13.592+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:13.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:13.964+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:13.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:13.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:14.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:14.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:14.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:14.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:14.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:14.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:14.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:14.072+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:14.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:14.072+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:14.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:14.076+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:15.193 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:36:15.201 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:36:15.206 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:36:15.223 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:36:15.516 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:15.512+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:16.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:16.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:16.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:16.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:16.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:16.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.112+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:16.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.112+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:16.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.156+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:16.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.168+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:36:16.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.168+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:36:16.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:16.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:16.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:16.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:16.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:16.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:16.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:16.698 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4117264617 2023-10-14T00:36:16.706 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:36:16.708 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243776 2023-10-14T00:36:16.708 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:36:16.709 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4117264617 ]] 2023-10-14T00:36:16.709 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4117264617 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\1\7\2\6\4\6\1\7 ]] 2023-10-14T00:36:16.709 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:36:17.152 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:17.148+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:36:17.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:17.164+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:17.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:17.164+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:19.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.392+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:19.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:19.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:19.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:19.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:19.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:19.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.460+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:19.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.460+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:19.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:19.492+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:20.732 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:20.724+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:20.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:20.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:20.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:20.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:20.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:20.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:20.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.800+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:20.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:20.808+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:21.019 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:21.016+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:22.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:22.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:22.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:22.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:22.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:22.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.272+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:22.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.272+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:22.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:22.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:23.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.688+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:23.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:23.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:23.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:23.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:23.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:23.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.716+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:23.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.716+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:23.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:23.716+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:24.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:24.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:24.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:24.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:24.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:24.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:24.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:24.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:24.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:24.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:26.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:26.028+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:26.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:26.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:26.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:26.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:26.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:26.048+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:26.717 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:36:26.719 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:36:26.720 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:36:26.737 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:36:26.813 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:26.808+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:36:26.846 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:26.844+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:28.088 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/2810373017 2023-10-14T00:36:28.094 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:36:28.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243788 2023-10-14T00:36:28.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:36:28.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/2810373017 ]] 2023-10-14T00:36:28.096 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/2810373017 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\1\1\7\2\6\4\6\1\7 ]] 2023-10-14T00:36:28.097 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/2810373017 2023-10-14T00:36:29.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:29.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:29.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:29.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:29.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:29.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.652+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:29.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.652+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:29.656 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.652+0000 7f660d1ed640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2749d40 handle_get_snapshots: failed to get snapshots: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:29.656+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:29.727 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.724+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2749e00 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.814 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.812+0000 7f660d1ed640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2748840 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.850 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.848+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb27489c0 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.879 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.876+0000 7f660d1ed640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2749800 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.887 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.884+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2748780 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.892 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.888+0000 7f660d1ed640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2748300 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.892 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.888+0000 7f660d9ee640 -1 librbd::mirror::GetInfoRequest: 0x55ccb27483c0 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.893 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.888+0000 7f660d1ed640 -1 librbd::mirror::GetInfoRequest: 0x55ccb2748a80 handle_get_mirror_image: failed to retrieve mirroring state: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:29.894 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:36:29.892+0000 7f660a9e8640 -1 librbd::api::Mirror: image_info_list: failed to list mirror image statuses: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:36:30.605 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/2810373017 until 2023-10-14T01:36:29.893743+0000 (3600 sec) 2023-10-14T00:36:30.658 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:36:30.681 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/2810373017 2023-10-14T00:36:31.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.664+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:31.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:31.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:31.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:31.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.676+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:31.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.828+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:31.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:31.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:31.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:31.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:31.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:31.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.936+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:31.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.936+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:31.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:31.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:31.970 INFO:tasks.workunit.client.0.smithi183.stderr:listed 32 entries 2023-10-14T00:36:32.020 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/2810373017 2023-10-14T00:36:32.020 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:36:32.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.660+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:32.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:32.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:32.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:32.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:32.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:32.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.736+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:32.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.736+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:32.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:32.776+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:34.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:34.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:34.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:34.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:34.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:34.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:34.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:34.884+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:35.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:35.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:36.615 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:36.608+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:36.616 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:36.612+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:36.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.648+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:36.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:36.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:36.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:36.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:36.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:36.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.668+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:36.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:36.700+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:36.910 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:36.908+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:37.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.848+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:37.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:37.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:37.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:37.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:37.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.868+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:37.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.924+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:37.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:37.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:37.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:37.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:37.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:37.968+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:38.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:38.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:38.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:38.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:38.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:38.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:38.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:38.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:38.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:38.964+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:38.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:38.964+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:38.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:38.968+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:39.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:39.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:39.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:39.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:39.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:39.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:39.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:39.544+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:39.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:39.544+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:41.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.492+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:41.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:41.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:41.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:41.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:41.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:41.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.576+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:41.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:41.588+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:42.042 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:36:42.042 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:36:42.044 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:36:42.078 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:36:42.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:42.372+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:42.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:42.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:42.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:42.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:42.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:42.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:42.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:42.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:42.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:42.568+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:43.061 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:43.060+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:43.536 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:36:43.539 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:36:43.540 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243803 2023-10-14T00:36:43.541 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:36:43.541 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:36:43.541 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\2\8\1\0\3\7\3\0\1\7 ]] 2023-10-14T00:36:43.541 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist add 172.21.15.183:0/4061210612 2023-10-14T00:36:43.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:43.932+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:43.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:43.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:43.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:43.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:43.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:43.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:43.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:43.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:43.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:43.980+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:43.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:43.980+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:44.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:44.004+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:44.614 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:44.616+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:45.297 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:45.300+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:45.298 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:45.300+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:45.463 INFO:tasks.workunit.client.0.smithi183.stderr:blocklisting 172.21.15.183:0/4061210612 until 2023-10-14T01:36:44.778025+0000 (3600 sec) 2023-10-14T00:36:45.563 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep -q 172.21.15.183:0/4061210612 2023-10-14T00:36:45.572 INFO:tasks.workunit.client.0.smithi183.stderr:+ ceph osd blocklist ls 2023-10-14T00:36:45.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:45.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:45.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:45.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:45.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:45.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:45.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:45.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:45.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:45.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:45.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:45.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:45.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:45.944+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:46.874 INFO:tasks.workunit.client.0.smithi183.stderr:listed 33 entries 2023-10-14T00:36:46.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ PREV_CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:36:46.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:36:46.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:46.976+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:46.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:46.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:46.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:46.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:47.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:47.020+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:47.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:47.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:47.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:47.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:47.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:47.028+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:47.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:47.028+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:47.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:47.032+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:48.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.663+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:48.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:48.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:48.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:48.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:48.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.723+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:48.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.723+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:48.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:48.735+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:48.994 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:48.995+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:50.192 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:50.191+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:50.192 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:50.191+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:51.948 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:51.947+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:51.959 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:51.959+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:52.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:52.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:52.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:52.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.095+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:52.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.095+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:52.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:52.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:52.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:52.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:52.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:52.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.747+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:52.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.747+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:52.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:52.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:53.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:53.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:53.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:53.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:53.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:53.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:53.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:53.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:54.023 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:36:54.023+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:36:54.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:54.507+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:36:54.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:54.507+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:54.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:54.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:55.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:55.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:55.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.027+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:55.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.031+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:55.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.031+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:55.544 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:55.543+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:36:55.556 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:55.555+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:36:55.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.835+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:36:55.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:55.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:55.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:55.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:55.859+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:56.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:56.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:56.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:56.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:56.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:56.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:56.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:56.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:56.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:56.675+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:56.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:56.675+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:56.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:56.687+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:56.938 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:36:56.942 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:36:56.951 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:36:56.957 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:36:57.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:36:57.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:36:57.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:36:57.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:36:57.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:36:57.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.195+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:36:57.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.195+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:36:57.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:36:57.211+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:36:58.145 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:36:58.161 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:36:58.162 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243818 2023-10-14T00:36:58.162 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:36:58.162 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:36:58.162 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:36:58.163 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:36:59.749 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:36:59.747+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.003+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.002 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.003+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2cbe280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.004 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.003+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.004 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.003+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3212000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.011+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.010 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.011+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2ffba80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.015+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.015 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.015+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb2e70480 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.022 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.023+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.022 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.023+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb295dd00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.028 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.027+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.028 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.027+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3257100 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.030 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.031+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.031 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.031+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb221f080 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.043+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.042 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.043+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb221e180 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.051+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.050 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.051+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3529880 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.053 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.055+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.053 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.055+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3257580 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.062 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.059+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.059+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb221ee80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.063+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.063 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.063+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb221f000 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.071+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.070 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.071+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb352ef00 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.075+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.075 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.075+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb352f400 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.080 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.079+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.081 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.079+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb3528500 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.087+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.087 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.087+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb37e2280 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.091+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.092 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.091+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb37e2900 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.098 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.099+0000 7f660e1ef640 -1 librbd::image::OpenRequest: failed to retrieve name: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:00.099 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T00:37:00.099+0000 7f660e1ef640 -1 librbd::ImageState: 0x55ccb37e2f80 failed to open image: (108) Cannot send after transport endpoint shutdown 2023-10-14T00:37:01.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.495+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:01.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:01.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:01.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:01.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:01.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:01.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.567+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:01.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.583+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:01.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:01.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:01.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:01.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:01.855+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:02.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:02.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:03.158 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:03.155+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:37:03.193 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:03.195+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:04.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:04.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:04.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:04.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:04.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:04.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:04.811+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:04.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:04.823+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:05.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.775+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:05.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:05.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:05.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:05.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:05.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:05.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.823+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:05.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.823+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:05.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:05.823+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:06.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.275+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:06.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:06.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:06.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:06.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:06.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.327+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:06.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:06.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:06.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:06.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:06.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.815+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:06.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:06.815+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:08.164 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:37:08.165 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:37:08.178 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:37:08.191 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:37:08.369 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:08.367+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:08.382 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:08.379+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:09.471 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:37:09.473 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:37:09.474 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243829 2023-10-14T00:37:09.474 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:37:09.474 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:37:09.474 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:37:09.475 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:37:09.740 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:09.739+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:11.591 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:11.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:12.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:12.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:12.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:12.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:12.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:12.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.791+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:12.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.791+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:12.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:12.807+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:13.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.579+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:37:13.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.579+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:37:13.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.591+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:13.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:13.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:13.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:13.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:13.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:13.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.667+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:13.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.667+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:13.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:13.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:15.760 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:15.759+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:15.780 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:15.779+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:16.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:16.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:16.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:16.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:16.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:16.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.499+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:16.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.499+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:16.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:16.527+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:16.616 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:16.615+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:37:16.616 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:16.615+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:18.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:18.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:18.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:18.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:18.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.127+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:18.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.127+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:18.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.183+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:18.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:18.431+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:19.478 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:37:19.478 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:37:19.485 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:37:19.486 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:37:19.663 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:19.663+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:20.496 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.495+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:20.522 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.523+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:20.655 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:37:20.655 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:37:20.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243840 2023-10-14T00:37:20.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:37:20.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:37:20.656 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:37:20.657 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:37:20.865 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:37:20.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.875+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:37:20.893 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.895+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:20.922 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.923+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:20.950 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.951+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:37:20.983 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.979+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:20.996 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:20.995+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:21.279 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:21.279+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:21.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:21.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:22.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.083+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:22.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:22.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:22.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:22.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:22.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:22.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.091+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:22.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.091+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:22.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.091+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:22.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.199+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:22.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:22.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:22.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:22.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.271+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:22.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.483+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:22.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.627+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:22.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:22.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:22.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:22.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.659+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:22.678 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:22.679+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:22.701 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:22.703+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:22.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:22.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:22.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:22.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:22.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:22.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.967+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:22.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.975+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:22.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:22.975+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:25.031 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:25.031+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:25.032 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:25.031+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:25.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:25.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:25.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:25.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:25.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.155+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:25.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.155+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:25.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.171+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:25.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:25.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:25.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:25.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:25.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:25.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:25.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:25.487+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:26.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:26.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:26.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:26.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:26.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:26.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.227+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:26.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.235+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:26.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:26.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:26.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:26.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.871+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:26.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:26.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:27.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:27.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:27.633 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:27.631+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:27.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:27.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:27.652 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:27.651+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:27.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:27.663+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:30.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:30.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:30.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.302+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:30.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:30.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:30.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.318+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:30.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.374+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:37:30.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.406+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:37:30.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:30.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:30.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:30.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:30.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:30.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:30.663 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:37:30.664 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:37:30.665 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:37:30.669 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:37:31.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:31.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:31.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:31.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:31.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:31.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.294+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:31.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.294+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:31.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.302+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:31.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:31.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:31.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:31.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:31.678+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:32.109 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:37:32.115 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:37:32.116 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243852 2023-10-14T00:37:32.116 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:37:32.116 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:37:32.117 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:37:32.117 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:37:32.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:32.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:32.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:32.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.186+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:32.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.186+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:32.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.186+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:32.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:32.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:32.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:32.514+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:35.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.098+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:37:35.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:35.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:35.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:35.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:35.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:35.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.158+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:35.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.158+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:35.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.158+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:35.223 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:35.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:35.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.322+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:35.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:35.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:35.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:35.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:35.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:35.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.354+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:35.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.354+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:35.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:35.362+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:36.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:36.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:36.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:36.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:36.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:36.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:36.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:36.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:36.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:36.746+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:36.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:36.758+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:36.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:36.766+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:40.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:40.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:40.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:40.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:40.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.186+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:40.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.186+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:40.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.186+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:40.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.990+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:40.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:40.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:41.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:40.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:41.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.010+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:41.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.010+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:41.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.010+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:41.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.010+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:41.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:41.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:41.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:41.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:41.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:41.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:41.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:41.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:41.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:41.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.742+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:41.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.742+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:41.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:41.742+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:42.119 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:37:42.121 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:37:42.125 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:37:42.131 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:37:43.517 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:37:43.518 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:37:43.519 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243863 2023-10-14T00:37:43.519 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:37:43.519 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:37:43.519 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:37:43.519 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:37:43.757 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:43.750+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:44.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:44.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:44.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:44.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.370+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:44.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.374+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:44.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.566+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:44.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:44.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:44.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:44.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.602+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:44.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.602+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:44.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.602+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:44.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:44.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.694+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:44.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.738+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:44.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:44.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:44.782+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:44.833 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:44.830+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:44.844 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:44.842+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:44.918 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:44.918+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:44.964 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:44.962+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:44.975 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:44.974+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:45.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.398+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:45.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:45.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:45.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:45.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:45.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:45.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.470+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:45.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.470+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:45.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:45.470+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:46.537 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:46.538+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:37:46.564 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:46.562+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:37:46.569 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:46.566+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:46.572 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:46.570+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:46.944 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:46.942+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:46.956 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:46.954+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:47.027 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:47.026+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:47.029 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:47.030+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:48.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.622+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:48.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:48.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:48.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:48.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:48.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:48.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.674+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:48.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.674+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:48.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:48.674+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:49.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.014+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:49.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:49.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:49.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:49.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:49.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:49.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.022+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:49.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.022+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:49.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.034+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:49.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:49.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:49.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:49.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:49.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:49.690+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:50.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.178+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:50.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:50.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:50.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:50.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:50.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:50.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.202+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:50.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:50.202+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:52.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.250+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:52.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:52.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:52.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:52.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:52.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:52.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.322+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:52.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.366+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:52.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.386+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:52.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:52.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:52.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:52.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:52.770+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:53.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:53.222+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:53.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:53.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:37:53.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:53.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:53.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:53.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:53.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:53.278+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:53.520 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:37:53.521 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:37:53.523 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:37:53.530 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:37:54.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:54.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:54.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:54.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:54.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:54.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:54.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:54.034+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:54.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:54.034+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:54.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:54.066+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:55.139 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:37:55.139 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:37:55.140 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243875 2023-10-14T00:37:55.140 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:37:55.140 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:37:55.140 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:37:55.141 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:37:56.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.282+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:37:56.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:56.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:56.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:56.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:56.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.394+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:37:56.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.394+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:37:56.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:56.394+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:56.913 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:56.906+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:37:56.942 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:37:56.938+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:37:58.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:58.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:58.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:58.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:58.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:58.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:58.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:58.378+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:37:59.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:59.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:37:59.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:59.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:37:59.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:59.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:37:59.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:59.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:37:59.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:37:59.606+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:00.051 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:00.050+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:00.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:00.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:00.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:00.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:00.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:00.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:00.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:00.330+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:01.595 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:01.594+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:01.735 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:01.734+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:02.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:02.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:02.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:02.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:02.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:02.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.254+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:02.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.254+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:02.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:02.270+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:05.149 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:38:05.150 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:38:05.152 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:38:05.165 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:38:06.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.074+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:06.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:06.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:06.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:06.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:06.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:06.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:06.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.130+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:06.318 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:38:06.318 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:38:06.322 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243886 2023-10-14T00:38:06.322 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:38:06.322 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:38:06.322 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:38:06.323 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:38:06.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:06.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:06.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:06.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:06.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.582+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:06.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.582+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:06.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.582+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:06.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.674+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:06.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:06.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:06.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:06.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:06.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.690+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:06.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.690+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:06.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.698+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:06.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.846+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:06.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:06.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:06.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:06.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:06.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.906+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:06.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:06.906+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:08.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:08.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:08.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:08.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.142+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:08.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.458+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:08.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:08.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:08.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:08.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:08.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:08.478+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:10.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:10.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:10.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:10.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:10.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:10.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:10.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:10.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:10.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:10.406+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:10.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:10.406+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:11.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:11.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:11.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:11.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:11.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.461+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:11.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:11.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:11.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:11.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:11.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.921+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:11.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.921+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:11.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:11.929+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:12.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.093+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:12.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:12.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:12.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:12.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:12.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.193+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:12.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.193+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:12.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.205+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:12.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:12.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:12.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:12.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:12.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:12.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.873+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:12.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.873+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:12.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:12.885+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:13.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:13.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:13.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:13.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:13.741+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:16.351 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:38:16.354 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:38:16.358 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:38:16.363 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:38:16.556 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:16.553+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:17.743 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:38:17.745 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:38:17.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243897 2023-10-14T00:38:17.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:38:17.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:38:17.750 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:38:17.750 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:38:18.225 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:18.217+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:18.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:18.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:18.895 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:18.893+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:18.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:18.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:18.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:18.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:18.917 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:18.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:18.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:18.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:18.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:18.945+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:18.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:18.945+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:18.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:18.973+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:20.168 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:20.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:20.202 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:20.201+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:21.911 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:21.909+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:22.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.697+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:22.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:22.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:22.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:22.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:22.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.761+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:22.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.761+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:22.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.761+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:22.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:22.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:22.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:22.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:22.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:22.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.833+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:22.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:22.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:22.882 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:22.881+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:22.919 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:22.913+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:22.930 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:22.929+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:23.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:23.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:23.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:23.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:23.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:23.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:23.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:23.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:23.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:23.669+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:23.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:23.669+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:23.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:23.689+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:23.855 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:23.853+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:25.134 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:25.133+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:25.140 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:25.137+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:25.140 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:25.137+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:25.534 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:25.533+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:25.535 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:25.533+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:26.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:26.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:26.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:26.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:26.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.069+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:26.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.085+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:26.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.293+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:26.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:26.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:26.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:26.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:26.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.357+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:26.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.357+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:26.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:26.357+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:27.751 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:38:27.752 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:38:27.754 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:38:27.773 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:38:29.303 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:38:29.303 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:38:29.304 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243909 2023-10-14T00:38:29.304 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:38:29.304 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:38:29.304 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:38:29.305 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:38:30.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.709+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:30.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:30.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:30.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:30.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:30.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:30.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.733+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:30.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.733+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:30.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.733+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:30.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.793+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:30.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:30.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:30.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:30.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:30.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:30.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:30.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:30.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:30.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:31.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:31.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:31.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:31.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:31.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:31.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:31.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:31.929+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:31.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:31.929+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:31.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:31.949+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:32.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.421+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:32.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:32.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:32.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:32.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:32.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:32.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.465+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:32.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.465+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:32.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:32.465+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:33.459 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:33.453+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:33.460 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:33.457+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:34.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:34.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:34.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:34.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:34.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:34.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:34.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:34.489+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:34.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:34.489+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:34.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:34.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:35.058 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:35.057+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:35.064 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:35.061+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:35.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.165+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:35.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:35.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:35.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:35.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:35.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.209+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:35.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.209+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:35.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:35.209+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:37.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.173+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:37.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:37.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:37.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:37.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:37.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:37.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.201+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:37.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.201+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:37.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.201+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:37.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.281+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:37.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:37.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:37.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:37.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:37.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:37.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.333+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:37.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:37.333+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:37.493 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:37.489+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:37.494 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:37.489+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:37.545 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:37.545+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:38.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:38.397+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:38.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:38.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:38.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:38.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:38.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:38.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:39.312 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:38:39.317 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:38:39.335 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:38:39.347 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:38:39.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:39.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:39.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:39.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:39.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:39.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:39.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:39.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:39.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:39.601+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:39.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:39.601+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:39.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:39.601+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:40.484 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:40.481+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:40.493 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:40.493+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:40.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:40.509+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:40.551 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:40.549+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:40.572 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:40.569+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:40.790 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:38:40.793 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:38:40.795 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243920 2023-10-14T00:38:40.795 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:38:40.796 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:38:40.796 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:38:40.796 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:38:40.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:40.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:40.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:40.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:40.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:40.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:40.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:40.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:40.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:40.973+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:40.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:40.973+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:41.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:40.989+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:42.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:42.937+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:43.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:43.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:43.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:43.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:43.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:43.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:43.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:43.005+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:43.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:43.005+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:43.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:43.005+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:44.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:44.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:44.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:44.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:44.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.245+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:44.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.245+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:44.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.245+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:44.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.721+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:44.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:44.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:44.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:44.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:44.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:44.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.821+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:44.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.821+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:44.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:44.821+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:45.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.225+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:45.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:45.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:45.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:45.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:45.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:45.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.269+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:45.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.269+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:45.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:45.285+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:47.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.429+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:47.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:47.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:47.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:47.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:47.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:47.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.457+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:47.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.457+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:47.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:47.481+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:48.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.641+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:38:48.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:48.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:48.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:48.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:48.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:48.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.681+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:48.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.681+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:48.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:48.681+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:50.253 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:50.253+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:50.798 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:38:50.803 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:38:50.808 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:38:50.834 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:38:51.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:51.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:51.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:51.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:51.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:51.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:51.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:51.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:51.618 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:51.617+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:51.653 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:51.649+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:51.653 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:51.649+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:52.114 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:38:52.114 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:38:52.115 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243932 2023-10-14T00:38:52.116 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:38:52.116 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:38:52.116 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:38:52.116 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:38:54.751 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:54.748+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:57.549 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:57.544+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:58.021 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:58.016+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:58.037 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:58.036+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:38:58.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:58.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:58.057 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:58.052+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:58.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:58.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:38:58.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:58.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:38:58.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:58.104+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:38:58.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:58.104+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:38:58.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:58.104+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:38:58.773 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:58.768+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:38:58.787 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:38:58.784+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:38:59.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:59.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:38:59.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:59.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:38:59.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:59.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:38:59.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:38:59.808+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:00.657 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:00.656+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:01.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.052+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:01.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.056+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:01.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.056+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:01.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:01.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:01.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:01.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:01.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:01.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.072+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:01.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.072+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:01.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.072+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:01.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:01.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:01.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:01.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:01.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:01.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.216+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:01.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.216+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:01.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.236+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:01.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:01.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:01.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:01.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:01.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:01.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.504+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:01.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.504+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:01.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:01.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:01.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:02.129 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:39:02.129 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:39:02.133 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:39:02.134 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:39:02.400 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:02.396+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:03.710 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:39:03.711 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:39:03.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243943 2023-10-14T00:39:03.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:39:03.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:39:03.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:39:03.713 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:39:04.095 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:04.092+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:04.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.204+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:04.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.220+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:04.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.260+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:04.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:04.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:04.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:04.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:04.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:04.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.304+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:04.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.304+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:04.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.304+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:04.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:04.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:04.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:04.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:04.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:04.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:04.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.396+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:04.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:04.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:04.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:04.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:04.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:04.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:04.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:04.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:04.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:04.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:04.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:04.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.664+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:04.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.664+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:04.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:04.668+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:05.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:05.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:05.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:05.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:05.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:05.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:05.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:05.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:05.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:05.716+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:05.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:05.716+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:05.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:05.728+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:07.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.548+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:07.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:07.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:07.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:07.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:07.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:07.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:07.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.584+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:07.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:07.592+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:09.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:09.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:09.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:09.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:09.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:09.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:09.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.480+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:09.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:09.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:10.893 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:10.892+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:11.342 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:11.340+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:39:11.355 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:11.352+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:11.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:11.836+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:11.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:11.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:11.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:11.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:11.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:11.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:11.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:11.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:11.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:11.980+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:11.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:11.992+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:12.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:12.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:12.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:12.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:12.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.344+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:12.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:12.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:12.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:12.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.592+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:12.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.592+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:12.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:12.592+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:13.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:13.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:13.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:13.204+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:13.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:13.212+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:13.720 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:39:13.721 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:39:13.721 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:39:13.740 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:39:15.193 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:39:15.197 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:39:15.198 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243955 2023-10-14T00:39:15.198 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:39:15.198 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:39:15.199 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:39:15.199 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:39:15.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.848+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:15.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:15.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:15.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:15.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:15.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:15.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.892+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:15.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.892+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:15.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.892+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:15.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:15.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:16.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.020+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:16.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:16.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:16.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.032+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:16.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.032+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:16.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:16.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:16.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:16.552+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:16.729 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:16.728+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:39:16.757 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:16.752+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:39:16.757 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:16.752+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:18.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.884+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:18.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:18.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:18.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:18.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:18.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:18.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.912+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:18.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.912+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:18.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:18.912+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:19.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:19.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:19.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:19.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:19.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:19.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.676+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:19.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.676+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:19.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:19.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:22.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.036+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:22.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:22.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:22.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:22.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:22.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.112+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:22.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.112+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:22.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.116+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:22.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:22.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:22.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:22.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:22.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.596+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:22.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.596+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:22.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:22.624+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:23.385 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:39:23.384+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:39:23.386 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:39:23.384+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:39:23.536 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:23.532+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:39:23.537 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:23.536+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:39:23.544 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:23.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:39:23.544 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:23.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:25.213 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:39:25.213 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:39:25.213 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:39:25.235 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:39:25.572 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:25.568+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:39:25.592 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:25.588+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:25.964 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:25.960+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:39:25.967 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:25.964+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:26.345 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:39:26.346 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:39:26.347 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243966 2023-10-14T00:39:26.347 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:39:26.347 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:39:26.347 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:39:26.348 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:39:27.261 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:27.256+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:27.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.268+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:27.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.268+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:27.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.304+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:27.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:27.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:27.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:27.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:27.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:27.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:27.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:27.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:27.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.744+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:27.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:27.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:27.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:27.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:27.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.768+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:27.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.768+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:27.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:27.768+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:29.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.080+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:29.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:29.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:29.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:29.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.112+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:29.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.348+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:29.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.928+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:29.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:29.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:29.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:29.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:29.996+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:31.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:31.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:31.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:31.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:31.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:31.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.508+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:31.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.508+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:31.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:31.512+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:32.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:32.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:32.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:32.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.364+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:32.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.364+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:32.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:32.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:32.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:32.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:32.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.496+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:32.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.496+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:32.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:32.496+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:34.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.264+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:34.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:34.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:34.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:34.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:34.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:34.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:34.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:34.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:34.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:34.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:34.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:34.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.844+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:34.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:34.844+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:35.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:35.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:35.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:35.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:35.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:35.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:35.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:35.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:35.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:35.332+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:35.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:35.332+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:35.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:35.348+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:36.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:36.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:36.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:36.228+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:36.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:36.228+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:36.350 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:39:36.350 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:39:36.352 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:39:36.373 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:39:37.028 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:37.028+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:39:37.029 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:37.028+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:37.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:37.468+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:37.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:37.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:37.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:37.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:37.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:37.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:37.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:37.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:37.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:37.524+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:37.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:37.540+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:37.818 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:39:37.820 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:39:37.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243977 2023-10-14T00:39:37.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:39:37.822 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:39:37.822 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:39:37.822 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:39:38.822 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:38.823+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:39:38.844 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:38.847+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:39:38.848 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:38.847+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:38.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:38.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:38.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:38.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:38.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:38.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:38.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:38.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:38.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:38.987+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:38.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:38.999+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:40.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:40.571+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:40.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:40.575+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:40.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:40.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:40.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:40.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:40.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:40.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:40.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:40.631+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:40.655 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:40.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:39:40.661 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:40.663+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:39:40.690 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:40.691+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:39:40.716 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:40.715+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:41.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:41.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:41.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:41.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:41.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:41.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.259+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:41.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.259+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:41.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.259+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:41.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.707+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:41.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:41.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:41.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:41.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:41.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.779+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:41.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.779+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:41.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:41.779+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:42.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:42.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:42.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:42.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:42.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.315+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:42.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.315+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:42.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.315+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:42.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.699+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:42.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:42.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:42.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:42.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:42.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:42.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:42.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:42.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:42.739+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:44.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:44.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:44.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:44.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:44.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:44.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.851+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:44.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.867+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:44.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.895+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:44.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.895+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:44.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:44.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:44.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.911+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:44.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.915+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:44.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.931+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:44.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:44.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:44.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:44.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:44.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:44.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.987+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:44.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.987+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:44.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:44.991+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:45.491 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:45.491+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:47.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:47.787+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:47.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:47.787+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:47.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:47.803+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:47.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:47.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:47.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:47.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:47.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:47.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:47.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:47.831+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:47.842 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:39:47.844 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:39:47.847 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:39:47.847 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:39:48.238 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:48.231+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:39:48.247 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:39:48.247+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:39:48.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:48.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:48.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:48.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:48.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.559+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:48.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.559+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:48.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.559+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:48.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.931+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:48.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:48.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:48.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:48.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:49.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:49.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.999+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:49.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.999+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:49.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:48.999+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:49.019 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:39:49.034 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:39:49.035 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697243989 2023-10-14T00:39:49.035 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:39:49.036 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:39:49.036 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:39:49.036 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:39:49.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.463+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:49.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:49.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:49.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:49.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:49.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:49.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.531+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:49.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.531+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:49.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.531+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:49.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.943+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:49.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:49.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:49.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:49.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:49.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.999+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:49.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.999+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:49.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:49.999+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:50.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:50.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:50.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:50.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:50.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:50.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:50.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.263+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:50.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.323+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:50.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:50.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:50.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:50.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:50.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:50.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.471+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:50.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.475+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:50.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.475+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:50.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.475+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:50.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.475+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:50.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:50.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:50.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:50.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:50.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:50.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.503+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:50.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.503+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:50.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:50.503+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:52.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.403+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:52.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:52.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:52.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:52.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:52.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.419+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:52.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.419+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:52.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.427+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:52.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:52.755+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:54.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:54.883+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:54.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:54.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:54.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:54.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:54.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:54.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:54.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:54.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:54.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:54.899+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:54.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:54.899+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:55.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:55.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:55.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.099+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:55.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.523+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:55.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:55.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:55.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:55.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:55.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:55.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:55.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:55.587+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:56.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.443+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:56.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:56.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:56.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:56.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:56.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:56.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.479+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:56.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.479+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:56.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:56.479+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:57.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.355+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:39:57.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.355+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:39:57.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.367+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:39:57.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:57.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:39:57.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:57.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:57.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:57.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:57.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:57.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:39:59.039 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:39:59.039 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:39:59.039 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:39:59.049 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:39:59.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:59.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:39:59.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:59.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:39:59.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:59.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:39:59.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:59.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:39:59.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:59.531+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:39:59.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:59.531+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:39:59.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:39:59.539+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:00.195 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:00.195+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:00.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.299+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:00.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:00.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:00.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:00.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:00.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:00.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.379+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:00.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.379+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:00.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:00.379+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:00.431 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:40:00.432 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:40:00.432 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244000 2023-10-14T00:40:00.432 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:40:00.432 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:40:00.433 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:40:00.433 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:40:01.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:01.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:01.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:01.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:01.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:01.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:01.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:01.579+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:01.779 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:01.779+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:01.908 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:01.907+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:02.392 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:02.391+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:02.433 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:02.435+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:02.444 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:02.443+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:40:02.445 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:02.443+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:02.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:02.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:02.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:02.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:02.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.663+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:02.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.663+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:02.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:02.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:02.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:02.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.731+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:02.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:02.731+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:03.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:03.335+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:40:03.357 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:03.359+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:05.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:05.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:05.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:05.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:05.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:05.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.643+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:05.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.643+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:05.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.651+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:05.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:05.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:05.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:05.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.875+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:05.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:05.875+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:07.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.687+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:07.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.687+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:40:07.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.707+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:07.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:07.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:07.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:07.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:07.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:07.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:07.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:07.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:08.263 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:08.263+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:08.269 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:08.271+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:40:08.269 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:08.271+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:09.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:09.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:09.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:09.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:09.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:09.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.183+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:09.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.183+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:09.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:09.191+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:09.364 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:09.363+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:09.861 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:40:09.863+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:40:09.867 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:40:09.867+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:40:09.933 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:09.935+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:40:09.941 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:09.943+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:10.440 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:40:10.440 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:40:10.447 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:40:10.463 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:40:10.504 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:10.499+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:11.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:11.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:11.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:11.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:11.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:11.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:11.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:11.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:11.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:11.443+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:11.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:11.451+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:11.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:11.451+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:11.798 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:40:11.798 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:40:11.800 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244011 2023-10-14T00:40:11.800 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:40:11.800 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:40:11.800 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:40:11.800 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:40:12.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:12.755+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:12.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:12.755+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:40:12.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:12.759+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:12.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:12.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:12.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:12.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:12.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:12.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:12.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:12.783+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:13.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:13.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:13.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:13.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:13.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:13.095+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:13.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:13.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:13.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:13.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:13.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:13.235+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:15.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:15.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:15.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:15.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:15.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:15.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:15.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:15.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:15.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:15.123+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:15.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:15.123+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:16.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:16.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:16.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:16.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:16.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:16.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:16.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:16.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:16.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:16.283+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:16.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:16.283+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:16.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:16.283+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:18.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:18.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:18.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:18.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:18.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:18.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.099+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:18.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.099+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:18.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.099+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:18.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.903+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:18.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:18.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.939+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:18.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:18.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:18.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.951+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:18.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:18.951+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:19.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:19.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:19.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:19.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:19.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:19.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:19.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:19.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:19.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:19.131+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:19.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:19.131+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:19.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:19.131+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:20.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:20.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:20.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:20.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:20.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.039+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:20.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.039+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:20.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.047+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:20.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.343+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:20.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:20.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:20.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:20.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:20.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:20.375+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:20.783 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:20.783+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:20.822 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:20.815+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:21.802 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:40:21.802 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:40:21.805 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:40:21.813 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:40:23.152 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:40:23.153 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:40:23.153 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244023 2023-10-14T00:40:23.153 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:40:23.153 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:40:23.154 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:40:23.154 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:40:24.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:24.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:24.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:24.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:24.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:24.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.274+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:24.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:24.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:24.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:24.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:25.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:25.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:25.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:25.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:25.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:25.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:25.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:25.242+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:25.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:25.242+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:25.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:25.270+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:26.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.650+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:26.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:26.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:26.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:26.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:26.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.682+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:26.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.682+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:26.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:26.682+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:27.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:27.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:27.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:27.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:27.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:27.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:27.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:27.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:28.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.874+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:28.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:28.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:28.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:28.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:28.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:28.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.930+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:28.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.930+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:28.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:28.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:29.005 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:29.002+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:40:29.037 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:29.034+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:29.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:29.354+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:29.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:29.362+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:29.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:29.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:29.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:29.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:29.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:29.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:29.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:29.390+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:31.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:31.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:31.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:31.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:31.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:31.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.486+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:31.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.486+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:31.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:31.494+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:32.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:32.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:32.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:32.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:32.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:32.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.090+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:32.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.090+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:32.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:32.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.350+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:32.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:32.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:32.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:32.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:32.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:32.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.394+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:32.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:32.394+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:33.181 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:40:33.182 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:40:33.191 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:40:33.192 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:40:33.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.354+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:33.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.354+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:40:33.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.366+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:33.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:33.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:33.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:33.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:33.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:33.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.434+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:33.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.434+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:33.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.450+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:33.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.490+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:33.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.490+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:40:33.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.506+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:33.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:33.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:33.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:33.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:33.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:33.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.558+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:33.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.558+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:33.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:33.558+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:34.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:34.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:34.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:34.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:34.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:34.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:34.591 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:40:34.601 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:40:34.602 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244034 2023-10-14T00:40:34.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:40:34.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:40:34.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:40:34.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:40:35.371 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:35.370+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:35.375 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:35.374+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:36.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.242+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:36.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.242+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:40:36.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.242+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:36.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:36.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:36.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:36.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:36.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:36.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.246+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:36.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.246+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:36.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:36.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:36.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:36.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:37.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.646+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:37.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:37.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:37.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:37.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:37.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:37.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.690+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:37.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.690+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:37.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:37.698+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:38.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.486+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:38.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.498+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:40:38.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.498+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:38.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:38.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:38.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:38.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:38.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:38.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.518+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:38.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.518+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:38.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:38.518+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:39.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:39.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:39.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:39.302+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:39.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:39.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:39.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:39.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:39.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:39.342+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:39.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:39.342+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:39.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:39.354+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:41.259 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:41.258+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:40:41.287 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:41.286+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:41.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.490+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:41.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:41.532 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:41.530+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:40:41.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:41.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:41.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:41.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:41.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.550+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:41.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.550+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:41.555 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:41.554+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:40:41.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.566+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:41.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.610+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:41.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:41.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:41.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:41.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:41.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:41.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.718+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:41.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.718+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:41.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:41.718+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:43.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.614+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:43.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:43.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:43.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:43.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:43.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.658+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:43.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.658+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:43.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:43.658+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:44.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:44.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:44.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:44.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:44.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:44.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:44.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:44.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:44.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:44.546+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:44.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:44.546+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:44.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:44.550+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:44.619 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:40:44.621 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:40:44.625 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:40:44.628 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:40:45.127 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:45.126+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:40:45.145 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:45.146+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:46.079 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:40:46.079 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:40:46.080 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244046 2023-10-14T00:40:46.080 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:40:46.080 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:40:46.080 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:40:46.080 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:40:46.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:46.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:46.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:46.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:46.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:46.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:46.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:46.630+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:46.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:46.630+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:46.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:46.642+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:47.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:47.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:47.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:47.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:47.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:47.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:47.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:47.514+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:50.246 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:50.246+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:52.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.330+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:52.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:52.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:52.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:52.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:52.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:52.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.365+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:52.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.365+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:52.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:52.365+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:53.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:53.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:53.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:53.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.429+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:53.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:53.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.929+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:53.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:53.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:53.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:53.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:53.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:53.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.961+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:53.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.961+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:53.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:53.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:54.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:54.061+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:54.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:54.945+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:54.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:54.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:54.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:54.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:54.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:54.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:54.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:54.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:55.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:55.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.029+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:55.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.029+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:55.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.045+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:55.216 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:40:55.213+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:40:55.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.289+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:55.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:55.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:55.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:55.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:55.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:55.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.297+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:55.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.297+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:55.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.297+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:55.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.469+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:55.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:55.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:55.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:55.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:55.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.505+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:55.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.833+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:55.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:55.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:55.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:55.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:55.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.869+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:55.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.869+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:55.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:55.885+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:56.090 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:40:56.094 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:40:56.101 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:40:56.105 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:40:56.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:56.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:56.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:56.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:56.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:56.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.185+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:56.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.185+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:56.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:56.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:56.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:56.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:56.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.225+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:40:56.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.229+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:56.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:56.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:56.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:56.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:56.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:56.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.265+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:56.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.265+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:56.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:56.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:57.286 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:40:57.287 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:40:57.288 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244057 2023-10-14T00:40:57.288 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:40:57.288 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:40:57.288 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:40:57.288 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:40:57.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:57.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:57.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:57.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:57.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:57.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:57.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:57.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.509+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:57.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:57.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:57.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:57.841+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:58.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.777+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:40:58.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:58.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:58.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:58.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:58.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:58.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:58.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:58.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:58.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:40:59.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:40:59.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:40:59.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:40:59.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:40:59.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:40:59.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:40:59.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:40:59.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:40:59.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:01.656 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:01.653+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:01.804 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:01.801+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:41:01.857 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:01.853+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:01.901 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:01.901+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:01.912 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:01.909+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:02.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:02.361+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:02.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:02.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:02.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:02.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:02.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:02.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:02.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:02.393+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:03.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.653+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:41:03.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.653+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:41:03.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.665+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:03.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:03.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:03.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:03.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:03.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:03.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.697+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:03.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.697+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:03.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:03.705+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:04.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.073+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:41:04.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.073+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:41:04.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.073+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:04.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:04.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:04.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:04.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:04.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:04.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.105+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:04.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.105+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:04.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:04.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:05.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:05.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:05.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:05.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.429+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:05.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.485+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:05.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:05.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:05.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:05.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:05.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:05.565+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:06.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:06.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:06.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:06.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:06.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:06.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:06.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:06.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:06.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:06.141+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:06.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:06.141+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:06.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:06.141+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:06.925 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:06.921+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:06.939 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:06.933+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:06.947 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:06.945+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:07.295 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:41:07.295 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:41:07.296 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:41:07.314 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:41:07.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:07.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:07.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:07.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:07.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:07.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.849+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:07.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.849+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:07.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:07.849+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:08.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:08.305+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:08.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:08.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:08.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:08.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:08.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:08.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:08.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:08.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:08.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:08.381+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:08.769 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:41:08.770 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:41:08.770 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244068 2023-10-14T00:41:08.770 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:41:08.771 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:41:08.771 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:41:08.771 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:41:09.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:09.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:09.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:09.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:09.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:09.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.589+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:09.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.589+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:09.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:09.605+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:10.345 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:10.341+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:10.376 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:10.373+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:10.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:10.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:10.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:10.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:10.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:10.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.793+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:10.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.793+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:10.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.793+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:10.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.809+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:41:10.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.809+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:41:10.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:10.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:10.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:10.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:11.587 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:11.585+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:11.602 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:11.601+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:11.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:11.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:11.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:11.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:11.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:11.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:11.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:11.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:11.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:11.993+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:11.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:11.993+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:13.172 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:13.169+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:13.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.169+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:13.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:13.181 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:13.181+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:13.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:13.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:13.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:13.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:13.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.209+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:13.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.209+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:13.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.209+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:13.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.465+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:41:13.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.533+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:13.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:13.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:13.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:13.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:13.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:13.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.569+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:13.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.569+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:13.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.589+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:13.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:13.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:13.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:13.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.829+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:13.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.829+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:13.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.869+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:13.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:13.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:13.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:13.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:14.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:14.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:14.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:14.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:14.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:14.001+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:14.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:14.001+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:14.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:14.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:17.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:17.925+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:17.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:17.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:17.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:17.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:18.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:18.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:18.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:18.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:18.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:18.001+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:18.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:18.001+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:18.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:18.009+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:18.548 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:18.545+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:18.779 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:41:18.780 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:41:18.785 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:41:18.793 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:41:18.974 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:18.973+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:19.001 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:19.001+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:19.002 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:19.001+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:19.958 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:41:19.962 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:41:19.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244079 2023-10-14T00:41:19.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:41:19.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:41:19.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:41:19.964 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:41:21.617 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:21.617+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:21.645 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:21.645+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:21.659 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:21.653+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:22.037 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:22.037+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:24.442 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:24.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:41:24.443 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:24.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:24.444 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:24.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:24.444 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:24.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:25.706 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:25.704+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:25.720 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:25.716+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:26.319 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:26.316+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:26.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:26.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:26.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:26.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:26.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:26.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:26.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:26.688+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:26.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:26.688+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:26.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:26.696+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:27.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:27.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:27.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:27.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:27.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:27.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:27.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:27.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:27.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:27.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:27.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:27.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:27.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.624+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:27.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.624+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:27.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:27.636+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:28.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:28.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:28.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:28.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:28.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:28.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:28.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:28.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:28.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:28.644+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:28.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:28.644+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:28.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:28.652+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:29.967 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:41:29.968 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:41:29.973 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:41:30.010 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:41:30.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:30.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:30.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:30.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.348+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:30.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:30.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:30.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:30.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:30.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.820+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:30.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.820+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:30.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:30.844+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:31.465 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:41:31.466 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:41:31.470 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244091 2023-10-14T00:41:31.471 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:41:31.471 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:41:31.471 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:41:31.471 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:41:31.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:31.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:31.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:31.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:31.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:31.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:31.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:31.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:31.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:31.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:31.936 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:31.932+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:31.955 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:31.952+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:32.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:32.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:32.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:32.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.124+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:32.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.124+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:32.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.128+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:32.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:32.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:32.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:32.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:32.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:32.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:32.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:33.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.336+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:33.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:33.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:33.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:33.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:33.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:33.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.348+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:33.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:33.348+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:33.774 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:33.772+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:34.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:34.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:34.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:34.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:34.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.428+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:34.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.428+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:34.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.456+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:34.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:34.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:34.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:34.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.756+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:34.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.756+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:34.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:34.756+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:36.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.568+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:36.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:36.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:36.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:36.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:36.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.584+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:36.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.584+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:36.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:36.584+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:38.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.340+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:38.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:38.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:38.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:38.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:38.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.380+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:38.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.380+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:38.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.392+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:38.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:38.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:38.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:38.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:38.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:38.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:38.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:38.908 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:38.904+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:38.917 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:38.916+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:39.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:39.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:39.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:39.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:39.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:39.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.220+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:39.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.220+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:39.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:39.220+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:40.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.388+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:40.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:40.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:40.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:40.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:40.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:40.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.504+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:40.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.504+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:40.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.504+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:40.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.524+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:40.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:40.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:40.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:40.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:40.596+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:41.472 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:41:41.473 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:41:41.473 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:41:41.495 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:41:42.738 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:41:42.738 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:41:42.739 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244102 2023-10-14T00:41:42.739 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:41:42.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:41:42.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:41:42.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:41:44.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.908+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:44.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:44.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:44.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:44.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:44.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:44.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.976+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:44.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.976+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:44.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:44.976+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:45.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.052+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:45.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:45.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:45.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:45.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:45.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.068+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:45.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.068+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:45.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:45.068+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:45.771 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:45.768+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:41:45.811 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:45.804+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:45.812 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:45.804+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:45.824 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:45.820+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:47.011 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:47.008+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:41:47.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.216+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:47.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:47.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:47.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:47.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:47.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:47.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:47.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.280+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:47.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.296+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:47.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.924+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:41:47.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.940+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:41:47.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.940+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:47.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:47.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:47.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:47.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:47.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:47.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.984+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:47.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.984+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:47.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:47.984+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:48.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.648+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:48.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:48.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:48.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:48.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:48.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:48.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.684+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:48.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:48.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:49.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:49.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:49.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:49.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:49.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:49.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:49.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:49.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:49.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:49.932+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:49.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:49.932+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:49.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:49.964+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:50.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:50.444+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:50.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:50.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:50.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:50.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:50.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:50.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:50.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:50.464+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:50.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:50.464+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:50.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:50.464+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:51.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.800+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:51.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:51.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:51.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:51.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:51.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.860+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:51.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.860+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:51.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:51.864+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:52.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:52.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:52.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:52.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:52.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:52.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:52.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:52.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:52.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:52.568+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:52.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:52.568+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:52.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:52.576+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:52.750 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:41:52.751 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:41:52.755 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:41:52.756 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:41:53.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:53.876+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:53.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:53.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:53.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:53.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:53.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:53.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:53.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:53.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:53.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:53.928+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:54.240 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:41:54.241 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:41:54.241 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244114 2023-10-14T00:41:54.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:41:54.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:41:54.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:41:54.242 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:41:54.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.292+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:54.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:54.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:54.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:54.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:54.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.352+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:54.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.352+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:54.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:54.352+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:55.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.300+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:41:55.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.328+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:55.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:55.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:55.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:55.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:55.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.384+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:55.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:55.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:55.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:55.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:55.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:55.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.964+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:55.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:55.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:56.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:56.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:56.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:56.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:56.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.024+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:56.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.024+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:56.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.040+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:56.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.879+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:41:56.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:41:56.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:56.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:56.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:56.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:56.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.975+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:56.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.975+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:56.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:56.975+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:57.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.015+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:41:57.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:57.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:57.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:57.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:57.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.059+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:57.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.059+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:57.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.059+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:57.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:57.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:57.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:57.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:57.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.579+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:41:57.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.579+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:41:57.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:57.579+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:41:57.865 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:57.863+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:41:57.883 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:41:57.883+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:41:58.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:58.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:41:58.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:58.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:41:58.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:58.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:41:58.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:58.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:41:58.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:41:58.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:00.431 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:00.431+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:01.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.359+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:01.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:01.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:01.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:01.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:01.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:01.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.443+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:01.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.443+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:01.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:01.443+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:02.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.539+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:42:02.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.539+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:42:02.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.575+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:02.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:02.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:02.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:02.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:02.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:02.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.631+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:02.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.631+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:02.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:02.631+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:03.809 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:03.807+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:42:03.813 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:03.811+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:03.814 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:03.811+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:42:04.243 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:42:04.248 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:42:04.248 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:42:04.270 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:42:05.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:05.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:05.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:05.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:05.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.171+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:05.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.175+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:05.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.295+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:05.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.311+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:05.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:05.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:05.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:05.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:05.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:05.331+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:05.666 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:42:05.667 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:42:05.667 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244125 2023-10-14T00:42:05.668 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:42:05.668 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:42:05.668 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:42:05.668 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:42:06.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:06.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.439+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:06.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.715+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:06.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:06.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:06.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:06.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:06.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:06.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.787+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:06.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.787+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:06.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.787+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:06.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:06.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:07.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:07.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:07.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.031+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:07.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:07.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:07.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.275+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:07.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.807+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:07.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:07.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:07.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:07.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:07.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.847+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:07.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.847+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:07.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:07.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:07.943+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:08.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:08.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:08.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:08.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:08.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:08.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:08.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:08.051+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:08.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:08.051+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:08.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:08.075+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:09.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:09.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:09.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:09.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:09.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.327+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:09.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.331+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:09.841 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:09.839+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:09.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.851+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:09.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:09.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:09.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:09.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:09.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.955+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:09.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.955+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:09.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:09.955+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:10.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.043+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:10.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:10.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:10.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:10.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:10.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:10.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.123+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:10.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.123+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:10.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.139+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:10.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.143+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:42:10.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.143+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:42:10.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.147+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:10.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:10.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:10.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:10.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.163+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:10.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.371+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:10.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:10.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:10.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:10.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.415+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:10.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.783+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:42:10.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.795+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:10.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:10.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:10.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:10.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:10.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:10.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.835+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:10.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.835+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:10.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:10.835+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:11.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:11.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:11.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:11.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:11.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:11.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:11.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:11.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:11.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:11.039+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:11.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:11.039+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:11.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:11.043+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:12.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:12.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:12.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:12.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:12.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:12.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:12.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:12.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:12.383+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:12.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:12.399+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:14.536 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:14.535+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:42:15.670 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:42:15.677 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:42:15.693 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:42:15.698 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:42:16.836 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:42:16.837 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:42:16.839 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244136 2023-10-14T00:42:16.839 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:42:16.839 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:42:16.839 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:42:16.839 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:42:17.482 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:17.483+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:42:17.503 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:17.503+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:42:22.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.411+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:42:22.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.411+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:42:22.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.423+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:22.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:22.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:22.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:22.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:22.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:22.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.447+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:22.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.447+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:22.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.447+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:22.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:22.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:22.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:22.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:22.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.763+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:22.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.763+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:22.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:22.763+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:23.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.667+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:42:23.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.667+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:42:23.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.675+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:23.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:23.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:23.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:23.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:23.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:23.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.727+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:23.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.727+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:23.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.735+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:23.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:23.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:23.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:23.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:23.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.979+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:23.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.979+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:23.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:23.979+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:24.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:24.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:24.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:24.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:24.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:24.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:24.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:24.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:24.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:24.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:24.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:24.155+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:24.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:24.155+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:26.843 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:42:26.844 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:42:26.844 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:42:26.853 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:42:28.158 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:42:28.158 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:42:28.159 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244148 2023-10-14T00:42:28.159 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:42:28.159 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:42:28.159 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:42:28.160 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:42:30.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:30.163+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:30.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:30.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:30.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:30.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:30.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:30.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:30.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:30.199+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:31.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.282+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:42:31.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.282+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:42:31.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.282+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:31.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:31.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:31.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:31.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:31.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:31.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:31.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:31.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:31.286+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:32.250 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:32.250+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:42:32.280 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:32.278+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:32.281 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:32.278+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:42:33.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.270+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:33.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:33.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.302+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:33.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:33.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:33.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:33.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.318+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:33.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.318+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:33.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.330+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:33.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:33.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:33.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:33.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:33.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:33.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.742+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:33.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.742+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:33.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:33.758+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:34.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:34.286+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:34.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:34.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:34.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:34.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:34.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:34.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:34.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:34.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:34.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:34.310+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:34.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:34.338+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:35.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.154+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:35.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:35.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:35.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.250+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:35.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.258+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:35.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:35.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:35.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.274+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:35.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:35.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.906+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:35.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:35.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:35.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:35.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:35.926+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:36.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:36.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:36.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:36.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:36.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:36.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:36.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:36.186+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:36.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:36.186+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:36.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:36.190+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:37.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:37.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:37.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:37.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:37.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:37.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:37.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:37.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:37.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:37.630+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:37.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:37.630+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:37.928 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:37.926+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:38.161 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:42:38.173 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:42:38.178 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:42:38.185 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:42:39.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:39.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:39.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:39.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:39.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:39.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.282+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:39.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.282+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:39.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:39.298+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:39.923 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:42:39.923 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:42:39.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244159 2023-10-14T00:42:39.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:42:39.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:42:39.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:42:39.924 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:42:41.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.086+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:41.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:41.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:41.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:41.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:41.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:41.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.114+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:41.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.114+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:41.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.486+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:41.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:41.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:41.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:41.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:41.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:41.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.582+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:41.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.582+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:41.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.582+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:41.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:41.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:42.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:42.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:42.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:42.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:42.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:42.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:42.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:42.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:42.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:42.082+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:42.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:42.082+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:42.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:42.086+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:43.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.610+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:43.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:43.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:43.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:43.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:43.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:43.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.738+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:43.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.738+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:43.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:43.758+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:44.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.330+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:44.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:44.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:44.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:44.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:44.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:44.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:44.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:44.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:44.438+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:45.725 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:45.722+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:45.741 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:45.742+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:42:47.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:46.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:47.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:47.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:47.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:47.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:47.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:47.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:47.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:47.046+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:47.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:47.050+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:47.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:47.050+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:48.155 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:48.146+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:42:48.174 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:48.170+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:42:48.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:48.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:48.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:48.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:48.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:48.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.758+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:48.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.758+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:48.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:48.762+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:49.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.698+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:49.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:49.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:49.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:49.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:49.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:49.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.734+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:49.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.734+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:49.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:49.746+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:49.934 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:42:49.935 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:42:49.944 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:42:49.961 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:42:51.435 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:42:51.436 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:42:51.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244171 2023-10-14T00:42:51.436 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:42:51.437 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:42:51.437 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:42:51.437 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:42:52.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:52.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:52.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:52.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:52.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:52.118+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:52.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:52.146+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:52.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:52.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:52.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:52.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:52.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:52.174+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:53.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.342+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:53.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.758+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:53.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:53.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:53.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:53.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:53.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:53.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:53.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:53.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:53.838+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:54.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:54.594+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:55.008 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:55.006+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:55.030 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:55.030+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:42:55.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.790+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:55.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:55.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:55.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:55.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:55.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.914+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:55.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.914+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:55.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.922+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:55.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.922+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:42:55.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.922+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:42:55.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.930+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:55.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:55.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:55.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:55.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:55.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:55.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.958+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:55.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.958+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:55.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:55.958+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:57.632 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:42:57.630+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:42:57.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.862+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:42:57.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:42:57.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:42:57.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:57.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:42:57.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:42:57.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.974+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:42:57.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.974+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:42:57.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:57.990+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:42:58.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:58.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:42:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:42:58.334+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:01.438 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:43:01.439 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:43:01.444 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:43:01.445 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:43:02.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:02.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:02.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:02.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:02.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:02.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.450+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:02.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.450+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:02.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:02.450+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:02.874 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:43:02.875 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:43:02.875 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244182 2023-10-14T00:43:02.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:43:02.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:43:02.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:43:02.876 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:43:05.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.789+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:05.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:05.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:05.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:05.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:05.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:05.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:05.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.833+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:05.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:05.877+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:06.168 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:06.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:06.504 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:06.501+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:06.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:06.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:06.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.537+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:06.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:06.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:06.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:06.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:06.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:06.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.909+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:06.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:06.921+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:07.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:07.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:07.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:07.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:07.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:07.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:07.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:07.829+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:07.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:07.829+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:09.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.453+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:09.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:09.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:09.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:09.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:09.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:09.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.513+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:09.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.513+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:09.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:09.513+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:10.014 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:10.009+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:43:10.015 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:10.013+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:10.232 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:10.229+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:43:10.254 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:10.245+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:12.884 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:43:12.889 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:43:12.896 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:43:12.900 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:43:13.331 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:13.329+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:13.356 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:13.353+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:14.369 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:43:14.369 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:43:14.370 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244194 2023-10-14T00:43:14.370 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:43:14.370 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:43:14.370 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:43:14.371 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:43:14.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.621+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:14.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:14.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:14.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.625+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:14.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:14.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:14.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:14.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:14.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.941+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:14.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:14.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:14.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:14.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.985+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:14.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.985+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:14.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:14.993+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:16.532 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:16.529+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:18.815 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:18.809+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:43:18.841 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:18.837+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:18.845 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:18.845+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:20.198 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:20.197+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:21.077 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:21.073+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:21.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:21.301+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:21.316 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:21.313+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:21.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.437+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:21.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.437+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:43:21.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.457+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:21.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:21.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:21.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:21.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:21.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:21.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:21.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:21.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.545+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:21.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:21.725+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:24.381 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:43:24.383 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:43:24.384 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:43:24.389 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:43:24.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.613+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:24.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:24.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:24.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:24.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:24.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:24.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.693+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:24.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.693+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:24.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:24.745+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:25.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:25.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:25.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:25.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.485+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:25.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.485+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:25.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:25.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:25.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:25.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:25.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:25.773 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:43:25.774 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:43:25.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244205 2023-10-14T00:43:25.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:43:25.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:43:25.775 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:43:25.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:43:25.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:25.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:25.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:25.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.805+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:25.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.825+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:25.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:25.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:25.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:25.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:25.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:25.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.901+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:25.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.901+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:25.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:25.913+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:26.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:26.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:26.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:26.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:26.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:26.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:26.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:26.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:26.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:26.061+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:26.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:26.061+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:26.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:26.089+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:27.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:27.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:27.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:27.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:27.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:27.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.745+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:27.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.745+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:27.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:27.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.873+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:27.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:27.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:27.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:27.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:27.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:27.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.897+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:27.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.897+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:27.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:27.921+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:28.099 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:28.097+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:28.108 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:28.105+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:28.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.209+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:28.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:28.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:28.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:28.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:28.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.257+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:28.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.257+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:28.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:28.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:30.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:30.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:30.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:30.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:30.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:30.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.217+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:30.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.229+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:30.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:30.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:30.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:30.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:30.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:30.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.697+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:30.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.697+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:30.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:30.697+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:31.618 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:31.617+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:32.288 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:32.285+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:43:32.295 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:32.293+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:33.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:33.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:33.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:33.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:33.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:33.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:33.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:33.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:33.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:33.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:33.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:33.349+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:34.810 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:34.809+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:35.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.397+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:35.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:35.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:35.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:35.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:35.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.437+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:35.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.437+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:35.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:35.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:35.788 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:43:35.790 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:43:35.790 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:43:35.791 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:43:35.909 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:35.905+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:43:36.018 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:36.005+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:43:36.018 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:36.009+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:36.046 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:36.045+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:36.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:36.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:36.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:36.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:36.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:36.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:36.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:36.337+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:36.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:36.337+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:37.204 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:43:37.207 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:43:37.209 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244217 2023-10-14T00:43:37.209 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:43:37.209 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:43:37.210 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:43:37.210 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:43:37.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:37.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:37.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:37.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:37.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:37.337+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:37.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:37.945+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:37.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:37.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:38.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:38.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:38.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:38.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.021+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:38.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.021+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:38.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.021+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:38.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.545+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:38.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.585+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:38.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:38.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:38.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:38.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.681+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:38.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.689+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:38.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.689+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:43:38.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.705+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:38.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:38.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:38.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:38.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:38.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:38.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.769+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:38.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.777+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:38.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.809+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:43:38.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.809+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:38.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:38.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:38.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:38.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:38.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:38.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.849+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:38.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.849+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:38.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:38.869+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:40.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:40.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:40.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:40.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:40.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:40.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:40.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:40.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:40.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:40.933+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:40.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:40.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:40.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:40.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:41.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:41.197+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:41.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:41.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:41.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:41.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:41.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:41.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:41.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:41.233+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:42.094 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:42.092+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:42.548 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:42.544+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:42.578 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:42.572+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:42.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.732+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:42.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:42.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:42.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:42.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:42.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.796+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:42.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.900+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:42.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.920+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:43:42.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.928+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:42.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:42.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:42.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:42.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:42.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:42.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:42.948+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:45.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:45.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:45.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:45.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:45.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:45.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.768+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:45.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.768+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:45.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:45.800+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:45.968 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:45.964+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:46.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:46.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:46.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:46.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:46.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:46.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.308+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:46.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.308+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:46.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.308+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:46.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:46.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:46.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:46.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:46.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.936+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:46.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:46.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:46.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:47.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:47.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:47.236 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:43:47.239 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:43:47.239 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:43:47.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:47.253 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:43:47.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.308+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:47.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:47.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:47.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:47.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:47.608+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:47.941 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:47.936+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:48.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:48.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:48.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:48.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:48.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.440+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:48.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.440+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:48.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.468+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:48.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:48.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:48.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:43:48.717 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:43:48.718 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244228 2023-10-14T00:43:48.718 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:43:48.718 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:43:48.718 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:43:48.719 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:43:48.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:48.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:48.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.764+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:48.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.764+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:48.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:48.776+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:49.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:49.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:49.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:49.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.168+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:49.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.348+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:49.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.356+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:43:49.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.384+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:49.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:49.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:49.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:49.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:49.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.452+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:43:49.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.452+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:43:49.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.476+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:49.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:49.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:49.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:49.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:49.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:52.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:52.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:52.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:52.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:52.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:52.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.224+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:52.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.224+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:52.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:52.224+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:52.601 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:52.596+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:53.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:53.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:53.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:53.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:53.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:53.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:53.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:53.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:53.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:53.752+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:53.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:53.752+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:53.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:53.752+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:54.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.948+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:54.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:54.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:54.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:54.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:54.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.976+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:54.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.976+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:54.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:54.976+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:55.191 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:55.188+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:43:55.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.236+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:55.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:55.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:55.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:55.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:55.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.284+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:55.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.284+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:55.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:55.288+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:57.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.156+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:57.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:57.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:57.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:57.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:57.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.172+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:57.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.172+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:57.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.188+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:57.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.528+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:43:57.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:57.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:57.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:57.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:57.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:57.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.596+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:57.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.596+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:57.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:57.660+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:57.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:43:57.888+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:43:58.730 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:43:58.731 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:43:58.757 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:43:58.770 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:43:58.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:58.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:58.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:58.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:59.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:59.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:59.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.012+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:59.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.012+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:59.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.036+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:43:59.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:43:59.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:43:59.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:43:59.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:43:59.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:43:59.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.848+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:43:59.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.848+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:43:59.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:43:59.884+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:00.307 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:44:00.309 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:44:00.312 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244240 2023-10-14T00:44:00.312 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:44:00.313 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:44:00.313 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:44:00.313 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:44:01.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.352+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:01.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:01.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:01.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:01.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:01.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:01.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.360+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:01.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.388+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:01.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:01.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:01.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:01.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:01.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.488+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:01.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.496+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:01.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:01.496+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:02.186 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:02.184+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:02.208 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:02.204+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:44:02.216 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:02.212+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:03.210 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:03.208+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:03.231 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:03.228+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:44:03.242 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:03.240+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:05.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:05.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:05.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:05.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:05.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:05.056+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:05.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:05.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:05.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:05.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:05.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:05.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:05.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:05.744+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:05.853 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:05.848+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:05.853 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:05.848+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:07.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.260+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:07.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:07.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:07.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:07.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:07.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:07.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.380+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:07.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.380+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:07.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:07.380+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:08.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:08.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:08.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:08.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:08.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.316+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:08.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.316+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:08.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.588+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:44:08.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.588+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:44:08.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.604+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:08.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:08.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:08.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:08.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:08.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:08.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.692+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:08.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.692+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:08.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:08.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:09.312 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:09.308+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:44:09.318 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:09.312+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:10.317 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:44:10.318 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:44:10.318 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:44:10.346 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:44:11.655 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:44:11.659 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:44:11.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244251 2023-10-14T00:44:11.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:44:11.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:44:11.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:44:11.662 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:44:12.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:12.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:12.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:12.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:12.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:12.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:12.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:12.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:12.708+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:13.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:13.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:13.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:13.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:13.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:13.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.364+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:13.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.364+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:13.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:13.368+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:15.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:15.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:15.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:15.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:15.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:15.120+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:15.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:15.132+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:15.905 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:15.904+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:15.906 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:15.904+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:16.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:16.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:16.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:16.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:16.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:16.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.180+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:16.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.180+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:16.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:16.212+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:16.978 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:16.976+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:17.011 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:17.012+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:17.877 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:17.876+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:44:18.059 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:18.060+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:18.085 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:18.084+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:19.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:19.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:19.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:19.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:19.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:19.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:19.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:19.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:19.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:19.463+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:19.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:19.467+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:20.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.051+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:20.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:20.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:20.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:20.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:20.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.063+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:20.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.063+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:20.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:20.075+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:21.667 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:44:21.667 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:44:21.667 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:44:21.686 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:44:22.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.579+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:22.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:22.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:22.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:22.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:22.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:22.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.619+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:22.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.619+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:22.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:22.627+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:23.197 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:44:23.200 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:44:23.201 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244263 2023-10-14T00:44:23.201 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:44:23.201 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:44:23.202 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:44:23.202 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:44:24.134 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:24.135+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:44:24.143 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:24.135+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:24.143 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:24.135+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:44:24.143 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:24.135+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:24.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.663+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:44:24.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.663+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:44:24.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.699+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:24.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:24.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:24.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:24.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:24.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:24.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.723+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:24.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.723+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:24.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.739+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:24.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.803+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:24.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:24.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:24.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:24.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:24.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:26.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:26.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:26.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:26.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:26.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:26.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:26.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:26.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:26.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:26.855+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:26.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:26.855+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:26.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:26.879+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:27.173 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:44:27.171+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:44:27.173 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:44:27.171+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:44:27.174 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:44:27.171+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T00:44:28.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:28.587+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:28.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:28.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:28.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:28.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:28.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:28.611+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:33.203 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:44:33.203 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:44:33.204 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:44:33.233 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:44:34.637 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:44:34.638 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:44:34.639 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244274 2023-10-14T00:44:34.639 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:44:34.640 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:44:34.640 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:44:34.640 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:44:42.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.547+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:42.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:42.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:42.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:42.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:42.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:42.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.591+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:42.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.591+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:42.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:42.615+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:43.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.235+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:43.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:43.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:43.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:43.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:43.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.351+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:43.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.351+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:43.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:43.371+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:44.598 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:44.599+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:44.661 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:44:44.662 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:44:44.666 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:44:44.686 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:44:44.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:44.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:44.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:44.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:44.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:44.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:44.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:44.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:45.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:45.307+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:45.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:45.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:45.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:45.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:45.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:45.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:45.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:45.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:45.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:45.347+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:46.090 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:44:46.090 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:44:46.092 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244286 2023-10-14T00:44:46.092 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:44:46.092 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:44:46.093 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:44:46.093 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:44:46.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.191+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:46.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:46.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:46.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:46.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:46.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.243+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:46.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.243+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:46.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:46.243+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:48.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:48.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:48.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:48.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:48.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:48.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:48.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:48.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:48.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:48.319+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:48.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:48.319+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:48.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:48.367+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:49.315 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:49.315+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:44:49.320 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:49.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:44:49.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:49.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:44:49.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:49.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:49.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:49.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:49.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:49.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:49.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:49.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:49.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:49.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:49.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:49.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:49.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:49.631+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:49.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:49.635+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:53.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.195+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:44:53.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.199+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:44:53.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.207+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:53.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:53.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:53.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:53.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:53.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:53.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.243+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:53.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.243+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:53.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:53.243+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:54.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:54.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:54.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:54.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:54.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:54.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:54.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:54.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:54.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:54.839+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:56.101 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:44:56.101 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:44:56.109 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:44:56.109 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:44:57.357 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:44:57.361 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:44:57.362 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244297 2023-10-14T00:44:57.363 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:44:57.363 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:44:57.363 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:44:57.363 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:44:57.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:57.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:57.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:57.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:57.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:57.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.451+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:57.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.451+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:57.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:57.455+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:58.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.446+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:44:58.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:58.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:58.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:58.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:58.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:58.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.518+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:58.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.518+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:58.549 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:58.546+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:44:58.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.554+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:44:58.565 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:44:58.562+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:44:58.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:44:58.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:44:58.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:44:58.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:44:58.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:44:58.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.694+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:44:58.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.694+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:44:58.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:44:58.718+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:00.866 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:00.862+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:45:00.895 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:00.894+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:01.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:01.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:01.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:01.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:01.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:01.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.270+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:01.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.270+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:01.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:01.270+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:02.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.450+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:02.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:02.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:02.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:02.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:02.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:02.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.502+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:02.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.502+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:02.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:02.502+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:05.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:05.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:05.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:05.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:05.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:05.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:05.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:05.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:05.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:05.058+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:05.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:05.066+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:05.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:05.066+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:07.366 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:45:07.367 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:45:07.367 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:45:07.370 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:45:08.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.134+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:08.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:08.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:08.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:08.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:08.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:08.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.218+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:08.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.218+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:08.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:08.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:08.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:08.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.810+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:08.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.810+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:08.852 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:45:08.852 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:45:08.852 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244308 2023-10-14T00:45:08.853 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:45:08.853 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:45:08.853 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:45:08.853 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:45:08.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:08.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:08.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:08.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.930+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:08.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:08.930+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:10.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:10.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:10.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:10.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:10.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:10.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:10.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:10.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:10.942+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:17.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:17.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:17.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:17.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:17.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:17.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:17.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:17.014+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:17.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:17.014+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:17.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:17.034+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:18.855 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:45:18.855 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:45:18.856 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:45:18.879 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:45:19.757 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:19.754+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:19.769 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:19.766+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:20.250 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:45:20.250 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:45:20.252 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244320 2023-10-14T00:45:20.253 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:45:20.253 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:45:20.253 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:45:20.253 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:45:21.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.106+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:21.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:21.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:21.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:21.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:21.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:21.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.134+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:21.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.134+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:21.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.146+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:21.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:21.202+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:22.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:22.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:22.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:22.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:22.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:22.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:22.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:22.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.078+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:22.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.362+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:22.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:22.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:22.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:22.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:22.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:22.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.426+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:22.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.426+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:22.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.438+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:22.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.798+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:22.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:22.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:22.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:22.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:22.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:22.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.846+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:22.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.846+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:22.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:22.858+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:23.001 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:22.998+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:45:23.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:23.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:23.027 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:23.026+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:23.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:23.038 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:23.038+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:23.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.042+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:23.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.626+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:23.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:23.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:23.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:23.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:23.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.666+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:23.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.666+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:23.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:23.666+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:23.850 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:23.850+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:23.866 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:23.862+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:24.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.198+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:24.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:24.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:24.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:24.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:24.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:24.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.234+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:24.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:24.238+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:26.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.558+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:26.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:26.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:26.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:26.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:26.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:26.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.606+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:26.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:26.606+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:27.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:27.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:27.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:27.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:27.310 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:27.302+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:45:27.310 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:27.302+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:27.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:27.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:28.780 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:28.778+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:28.828 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:28.826+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:29.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:29.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:29.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:29.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:29.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:29.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:29.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:29.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:29.686+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:30.261 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:45:30.261 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:45:30.262 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:45:30.312 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:45:31.357 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:45:31.357 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:45:31.359 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244331 2023-10-14T00:45:31.359 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:45:31.359 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:45:31.359 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:45:31.360 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:45:31.953 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:31.950+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:33.202 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:33.202+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:34.703 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:34.702+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:45:34.725 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:34.722+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:34.737 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:34.730+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:35.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.634+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:45:35.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.634+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:45:35.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.658+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:35.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:35.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:35.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:35.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:35.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:35.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.674+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:35.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.674+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:35.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:35.674+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:36.340 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:36.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:36.347 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:36.346+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:36.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:36.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:36.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:36.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:36.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.450+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:36.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:36.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:36.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:36.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:36.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:36.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.958+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:36.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:36.958+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:37.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.738+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:37.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:37.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:37.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:37.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:37.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:37.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.782+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:37.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.790+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:37.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:37.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:39.752 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:39.749+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:39.765 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:39.761+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:40.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:40.973+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:40.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:40.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:40.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:40.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:41.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:41.005+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:41.362 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:45:41.364 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:45:41.380 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:45:41.382 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:45:42.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.069+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:42.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:42.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:42.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:42.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:42.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:42.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.129+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:42.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.129+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:42.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:42.137+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:42.635 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:42.633+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:45:42.647 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:42.645+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:42.694 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:42.693+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:42.994 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:45:42.996 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:45:42.997 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244342 2023-10-14T00:45:42.997 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:45:42.997 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:45:42.998 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:45:42.998 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:45:45.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.561+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:45:45.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.561+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:45:45.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.581+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:45.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.585+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:45.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:45.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:45.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:45.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:45.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.625+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:45.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.625+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:45.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:45.625+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:46.542 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:46.537+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:48.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:48.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:48.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:48.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:48.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.313+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:48.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.313+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:48.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.313+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:48.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:48.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:48.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:48.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:48.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:48.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.485+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:48.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:48.497+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:49.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.025+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:49.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:49.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:49.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:49.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:49.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:49.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.077+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:49.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.077+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:49.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:49.097+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:50.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.117+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:50.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:50.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:50.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:50.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:50.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.157+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:50.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:50.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:50.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:50.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:50.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.281+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:50.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.281+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:50.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:50.289+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:52.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:52.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:52.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:52.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:52.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:52.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:52.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:52.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:52.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:52.361+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:52.505 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:52.501+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:45:52.515 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:52.513+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:53.005 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:45:53.008 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:45:53.008 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:45:53.010 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:45:53.245 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:53.241+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:45:53.259 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:53.257+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:45:53.275 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:53.273+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:45:54.572 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:45:54.572 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:45:54.573 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244354 2023-10-14T00:45:54.573 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:45:54.573 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:45:54.573 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:45:54.574 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:45:54.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:54.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:54.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:54.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:54.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:54.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:54.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:54.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:54.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.933+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:54.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.933+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:55.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:54.997+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:55.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.617+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:55.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:55.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:55.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:55.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:55.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.669+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:55.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.669+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:55.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.669+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:55.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.921+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:55.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:55.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:55.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:55.965+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:56.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.393+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:45:56.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:45:56.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:56.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:56.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:56.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:56.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.477+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:56.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.477+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:56.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:56.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:58.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:45:58.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:58.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:58.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:58.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.373+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:58.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.393+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:58.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.393+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:58.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:45:58.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:45:58.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:45:58.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.557+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:45:58.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.557+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:45:58.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:45:58.577+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:45:59.259 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:45:59.253+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:00.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:00.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:00.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:00.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:00.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:00.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.617+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:00.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.617+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:00.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:00.645+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:01.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:01.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:01.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:01.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:01.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:01.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.749+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:01.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.749+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:01.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.749+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:01.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:01.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:01.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:01.901+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:02.026 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:02.025+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:03.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:03.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:03.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:03.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:03.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:03.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:03.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:03.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:03.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:03.129+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:03.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:03.129+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:03.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:03.129+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:04.579 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:46:04.580 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:46:04.580 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:46:04.581 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:46:05.160 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:05.157+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:05.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:05.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:06.040 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:46:06.045 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:46:06.053 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244366 2023-10-14T00:46:06.053 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:46:06.053 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:46:06.053 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:46:06.053 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:46:07.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:07.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:07.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:07.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:07.446 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:07.445+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:07.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:07.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:07.452 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:07.449+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:07.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:07.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:07.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:07.453+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:07.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:07.453+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:07.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:07.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:08.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:08.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:08.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:08.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:08.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:08.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:08.590 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:08.589+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:46:08.591 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:08.589+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:08.591 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:08.589+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:09.424 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:09.421+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:09.433 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:09.429+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:10.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:10.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:10.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:10.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:10.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:10.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.928+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:10.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.928+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:10.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:10.948+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:11.733 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:11.728+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:12.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.180+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:12.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:12.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:12.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:12.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:12.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:12.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.244+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:12.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.244+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:12.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:12.244+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:14.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.300+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:14.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:14.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:14.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:14.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:14.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:14.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.412+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:14.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.412+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:14.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.412+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:14.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.572+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:14.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:14.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:14.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:14.604+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:15.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.608+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:15.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:15.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:15.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:15.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:15.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.676+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:15.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.676+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:15.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:15.676+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:16.061 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:46:16.062 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:46:16.066 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:46:16.074 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:46:17.518 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:17.516+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:17.551 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:17.548+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:17.729 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:46:17.742 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:46:17.744 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244377 2023-10-14T00:46:17.744 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:46:17.744 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:46:17.744 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:46:17.745 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:46:19.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:19.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:19.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:19.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:19.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:19.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:19.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:19.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:19.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:19.940+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:23.262 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:23.260+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:24.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:24.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:24.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:24.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:24.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.320+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:24.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.320+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:24.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.320+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:24.391 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:24.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:24.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:24.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:24.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:24.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:24.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.792+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:24.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.792+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:24.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:24.808+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:27.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.608+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:27.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:27.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:27.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:27.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:27.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:27.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.684+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:27.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.684+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:27.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:27.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.704+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:27.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:27.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:27.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:27.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:27.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.736+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:27.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.736+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:27.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:27.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:27.745 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:46:27.756 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:46:27.764 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:46:27.767 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:46:28.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:28.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:28.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:28.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.260+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:28.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.260+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:28.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.260+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:28.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.460+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:28.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:28.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:28.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:28.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:28.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:28.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.528+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:28.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.528+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:28.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:28.528+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:29.061 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:46:29.069 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:46:29.070 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244389 2023-10-14T00:46:29.071 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:46:29.071 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:46:29.071 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:46:29.071 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:46:29.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:29.360+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:31.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.672+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:31.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:31.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:31.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:31.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:31.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:31.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.736+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:31.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.736+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:31.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:31.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:32.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:32.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:32.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:32.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:32.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:32.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:32.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:32.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:32.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:32.428+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:32.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:32.428+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:32.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:32.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:32.961 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:32.956+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:46:32.968 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:32.964+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:33.346 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:33.344+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:46:33.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:33.356+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:33.389 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:33.384+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:34.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:34.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:34.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:34.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:34.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:34.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.112+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:34.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.124+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:34.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.140+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:34.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:34.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:34.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:34.816+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:35.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.260+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:35.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:35.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:35.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.316+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:35.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.936+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:35.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:35.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:35.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:35.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:35.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.976+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:35.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.976+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:35.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:35.976+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:37.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.196+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:37.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:37.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:37.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.224+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:37.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:37.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:37.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:37.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.380+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:37.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:37.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.628+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:46:37.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.632+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:37.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:37.640 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:37.640+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:37.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:37.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:37.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:37.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:37.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.684+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:37.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.684+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:37.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.684+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:37.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.712+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:46:37.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.712+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:46:37.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.712+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:37.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:37.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:37.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:37.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:37.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:37.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.748+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:37.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.748+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:37.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.764+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:37.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:37.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:37.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:37.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:39.039 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:39.040+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:39.089 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:46:39.094 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:46:39.099 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:46:39.102 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:46:40.187 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:40.188+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:46:40.194 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:40.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:46:40.195 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:40.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:40.200 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:40.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:40.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:40.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:40.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:40.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:40.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.604+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:40.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:40.686 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:46:40.686 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:46:40.687 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244400 2023-10-14T00:46:40.687 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:46:40.687 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:46:40.688 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:46:40.688 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:46:40.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:40.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:40.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.704+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:40.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.704+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:40.777 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:40.776+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:40.781 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:40.780+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:40.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:40.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:40.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:40.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:40.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:40.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:40.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:41.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:41.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:41.040+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:41.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:41.040+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:41.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:41.068+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:42.024 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:42.024+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:42.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:42.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:42.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:42.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:42.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:42.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:42.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:42.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:42.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:42.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:42.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:42.564+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:42.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:42.608+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:43.275 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:43.276+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:43.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:43.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:43.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:43.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:43.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:43.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:43.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:43.912+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:43.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:43.912+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:44.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:44.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:44.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:44.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:44.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:44.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:44.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:44.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:44.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:44.104+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:44.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:44.104+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:44.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:44.124+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:45.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:45.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:45.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:45.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:45.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.287+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:45.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.287+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:45.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:45.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:45.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:45.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:45.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:45.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:45.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:45.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:46.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:46.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:46.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.999+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:46.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.999+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:46.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:45.999+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:46.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:46.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:46.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:46.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:46.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:46.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:46.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:46.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:46.500 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:46.499+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:46:46.513 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:46.511+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:46.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:46.515+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:48.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.551+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:48.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:48.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:48.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:48.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:48.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:48.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.619+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:48.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.619+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:48.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:48.619+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:49.053 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:49.051+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:50.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.567+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:50.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:50.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:50.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:50.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.587+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.587+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:50.587+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:50.708 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:46:50.708 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:46:50.709 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:46:50.729 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:46:50.881 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:50.879+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:46:50.905 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:50.903+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:51.434 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:46:51.431+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:46:52.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.223+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:46:52.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.223+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:46:52.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.231+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:52.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:52.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:52.313 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:46:52.314 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:46:52.314 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244412 2023-10-14T00:46:52.314 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:46:52.315 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:46:52.315 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:46:52.315 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:46:52.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:52.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:52.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:52.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.327+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:52.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.327+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:52.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.335+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:52.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.343+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:46:52.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.343+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:46:52.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.391+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:52.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:52.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:52.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:52.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:52.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:52.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.431+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:52.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.431+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:52.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:52.443+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:53.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.255+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:53.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:53.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:53.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:53.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:53.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:53.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.291+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:53.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.291+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:53.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:53.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:53.635 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:46:53.635+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:46:53.635 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:46:53.635+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:46:54.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:54.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:54.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:54.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:54.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.703+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:54.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.703+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:54.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.703+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:54.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:54.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:54.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:54.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:54.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.799+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:54.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.815+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:54.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:54.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:46:54.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:54.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:54.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:54.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:54.907+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:55.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:55.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:55.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:55.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:55.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:55.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:55.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:55.863+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:55.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:55.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:46:57.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.507+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:46:57.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:46:57.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:46:57.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:46:57.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:46:57.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.543+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:46:57.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.543+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:46:57.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:46:57.555+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:00.092 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:00.091+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:00.286 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:00.287+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:47:00.316 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:00.315+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:00.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:00.975+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:00.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:00.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:00.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:00.995+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:01.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:01.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:01.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.003+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:01.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.003+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:01.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.027+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:01.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:01.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:01.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:01.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:01.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:01.575+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:02.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:02.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:02.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:02.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:02.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:02.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.107+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:02.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.107+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:02.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:02.107+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:02.321 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:47:02.322 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:47:02.329 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:47:02.336 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:47:03.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:03.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:03.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:03.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:03.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:03.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:03.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:03.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.611+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:03.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:03.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:03.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:03.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:03.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:03.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.655+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:03.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.655+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:03.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.655+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:03.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.879+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:03.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:03.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:03.913 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:47:03.913 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:47:03.914 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244423 2023-10-14T00:47:03.914 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:47:03.915 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:47:03.915 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:47:03.915 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:47:03.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:03.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:03.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:03.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:03.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:03.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:03.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.971+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:03.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:03.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:04.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:04.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:04.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:04.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.015+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:04.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.015+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:04.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.015+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:04.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:04.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:04.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:04.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:04.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.855+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:04.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.855+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:04.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:04.855+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:05.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:05.079+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:05.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:05.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:05.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:05.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:05.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:05.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:05.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:05.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:05.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:05.283+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:05.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:05.283+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:05.302 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:05.303+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:07.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:07.075+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:07.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:07.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:07.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:07.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:07.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:07.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:07.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:07.111+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:08.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.239+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:08.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:08.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:08.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:08.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:08.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:08.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.319+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:08.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.319+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:08.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.327+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:08.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:08.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:08.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:08.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:08.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:08.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.771+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:08.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.771+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:08.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:08.779+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:09.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:09.375+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:09.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:09.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:09.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:09.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:09.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:09.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:09.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:09.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:09.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:09.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:09.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:09.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:12.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.603+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:12.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:12.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:12.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:12.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:12.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:12.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.607+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:12.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.607+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:12.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:12.611+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:13.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.635+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:13.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:13.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:13.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:13.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:13.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:13.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.715+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:13.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.715+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:13.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.715+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:13.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.871+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:13.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:13.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:13.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:13.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:13.920 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:47:13.920 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:47:13.929 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:47:13.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:13.939+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:13.941 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:47:15.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:15.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:15.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:15.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:15.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:15.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:15.282 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:47:15.282 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:47:15.283 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244435 2023-10-14T00:47:15.283 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:47:15.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:47:15.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:47:15.284 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:47:15.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:15.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:15.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:15.299+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:15.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:15.299+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:15.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:15.299+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:16.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:16.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:16.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:16.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.475+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:16.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.475+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:16.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.483+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:16.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:16.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:16.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:16.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:16.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:16.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.835+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:16.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:16.835+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:18.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:18.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:18.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:18.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:18.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:18.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:18.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:18.643+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:18.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:18.959+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:18.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:18.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:19.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:19.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:19.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:19.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:19.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:19.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:19.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:19.031+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:19.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:19.031+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:19.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:19.043+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:19.210 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:19.211+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:19.229 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:19.227+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:19.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:19.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:20.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.718+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:20.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:20.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:20.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:20.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:20.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:20.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.750+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:20.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:20.750+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:22.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.266+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:22.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:22.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:22.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:22.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:22.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:22.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.366+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:22.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.366+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:22.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:22.366+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:23.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:23.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:23.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:23.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:23.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:23.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.654+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:23.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.654+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:23.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:23.654+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:24.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.466+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:24.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:24.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:24.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:24.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:24.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.538+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:24.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.538+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:24.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:24.538+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:24.994 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:24.990+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:25.322 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:47:25.329 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:47:25.333 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:47:25.348 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:47:26.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.158+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:26.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:26.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:26.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.238+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:26.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.238+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:26.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:26.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:26.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:26.262+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:26.826 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:47:26.826 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:47:26.828 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244446 2023-10-14T00:47:26.828 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:47:26.828 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:47:26.828 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:47:26.829 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:47:27.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:27.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:27.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:27.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:27.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:27.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.354+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:27.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.354+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:27.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:27.370+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:27.511 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:27.510+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:47:27.518 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:27.518+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:27.531 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:27.530+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:28.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.462+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:28.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:28.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:28.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:28.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.466+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:28.541 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:28.538+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:47:28.551 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:28.550+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:28.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.850+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:28.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:28.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:28.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:28.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:28.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:28.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.894+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:28.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.902+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:28.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:28.906+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:28.942 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:28.942+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:30.323 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:30.322+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:47:30.334 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:30.330+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:30.908 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:30.906+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:47:30.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:30.942+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:30.955 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:30.954+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:31.504 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:31.502+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:31.504 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:31.502+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:31.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:31.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:31.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:31.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:31.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:31.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:31.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:31.694+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:31.789 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:31.786+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:32.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:32.786+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:32.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:32.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:32.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:32.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:32.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:32.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:32.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:32.818+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:32.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:32.818+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:35.147 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:35.142+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:47:35.154 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:35.154+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:35.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:35.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:35.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:35.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:36.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.334+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:36.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:36.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:36.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:36.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:36.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:36.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.514+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:36.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.514+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:36.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:36.542+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:36.832 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:47:36.833 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:47:36.834 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:47:36.849 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:47:37.452 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:37.450+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:37.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.642+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:47:37.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.670+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:37.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:37.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:37.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:37.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:37.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:37.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.706+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:37.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.706+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:37.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:37.738+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:37.961 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:47:37.962 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:47:37.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244457 2023-10-14T00:47:37.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:47:37.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:47:37.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:47:37.963 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:47:38.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.010+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:38.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:38.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:38.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:38.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:38.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:38.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:38.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:38.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:38.918+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:39.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:39.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:39.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:39.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.086+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:39.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.086+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:39.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.142+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:39.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:39.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:39.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:39.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.598+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:39.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.598+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:39.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:39.602+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:42.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:42.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:42.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:42.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:42.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:42.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:42.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:42.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:42.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:42.850+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:42.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:42.850+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:42.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:42.858+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:43.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.454+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:43.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:43.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:43.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:43.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:43.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.522+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:43.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.522+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:43.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.522+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:43.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:43.522+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:45.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:45.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:45.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:45.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:45.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:45.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:45.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:45.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:45.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:45.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:45.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:45.642+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:45.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:45.642+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:46.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:46.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:46.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:46.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:46.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:46.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.214+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:46.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.214+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:46.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.222+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:46.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.306+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:47:46.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:46.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:46.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:46.463 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:46.462+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:47.473 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:47.470+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:48.007 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:47:48.007 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:47:48.009 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:47:48.015 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:47:48.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:48.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:48.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:48.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:48.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:48.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.826+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:48.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.826+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:48.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:48.842+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:49.353 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:47:49.357 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:47:49.361 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244469 2023-10-14T00:47:49.361 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:47:49.361 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:47:49.361 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:47:49.362 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:47:50.049 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:50.046+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:50.175 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:50.174+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:50.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.746+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:50.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:50.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:50.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:50.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:50.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:50.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.782+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:50.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:50.802+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:51.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:51.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:51.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.238+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:51.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.466+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:51.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:51.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:51.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:51.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:51.518+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:52.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.026+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:52.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:52.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:52.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:52.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:52.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.070+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:52.088 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:52.082+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:47:52.124 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:47:52.122+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:47:52.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.138+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:52.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:52.142+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:55.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:55.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:55.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:55.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:55.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:55.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:55.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:55.378+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:55.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:55.378+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:57.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.069+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:57.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:57.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:57.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:57.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.141+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:57.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.245+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:57.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:57.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:57.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:57.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:57.325+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:58.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.893+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:47:58.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:47:58.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:58.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:58.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:58.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.933+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:58.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.933+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:58.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:58.945+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:59.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:59.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:47:59.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:59.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:47:59.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:59.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:47:59.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:59.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:47:59.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:59.317+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:47:59.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:59.317+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:47:59.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:47:59.317+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:47:59.363 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:47:59.363 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:47:59.364 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:47:59.364 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:48:00.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:00.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:00.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:00.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:00.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:00.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.757+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:00.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.757+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:00.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:00.781+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:00.821 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:48:00.837 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:48:00.838 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244480 2023-10-14T00:48:00.838 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:48:00.838 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:48:00.839 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:48:00.839 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:48:03.056 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:48:03.053+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:48:03.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.705+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:48:03.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.705+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:48:03.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.705+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:03.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:03.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:03.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:03.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:03.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:03.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.709+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:03.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.709+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:03.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:03.709+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:04.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:04.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:05.347 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:05.345+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:48:05.370 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:05.365+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:48:08.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:08.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:08.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:08.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:08.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.225+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:08.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.225+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:08.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.225+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:08.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:08.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:08.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:08.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:08.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.309+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:08.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.309+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:08.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:08.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:08.918 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:08.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:09.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.033+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:09.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:09.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:09.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:09.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:09.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:09.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.081+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:09.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.081+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:09.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:09.089+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:10.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:10.305 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:10.301+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:10.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:10.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:10.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:10.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:10.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.373+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:10.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.373+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:10.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:10.385+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:10.649 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:10.645+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:48:10.666 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:10.665+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:10.840 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:48:10.840 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:48:10.840 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:48:10.857 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:48:11.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.013+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:11.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:11.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:11.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:11.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:11.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:11.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.101+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:11.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.109+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:11.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.109+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:11.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:11.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:11.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:11.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:11.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:11.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:12.369 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:48:12.369 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:48:12.378 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244492 2023-10-14T00:48:12.378 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:48:12.378 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:48:12.379 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:48:12.379 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:48:12.466 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:48:12.457+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:48:12.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:12.777+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:12.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:12.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:12.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:12.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:12.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:12.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:12.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:12.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:12.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:12.797+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:12.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:12.857+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:14.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:14.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:14.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:14.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.261+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.261+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:14.261+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:15.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.365+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:15.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:15.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:15.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:15.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:15.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:15.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:15.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:15.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.413+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:15.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:15.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:15.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:15.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:15.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.741+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:15.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.741+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:15.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.745+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:15.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:15.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:15.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:15.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.789+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:15.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.789+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:15.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:15.793+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:16.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.885+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:16.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:16.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:16.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:16.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:16.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:16.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.945+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:16.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.949+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:16.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:16.949+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:17.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:17.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:17.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:17.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:17.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:17.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.277+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:17.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.277+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:17.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.277+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:17.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.441+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:48:17.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.441+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:48:17.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:17.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:17.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:17.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:17.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.805+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:17.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:17.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:17.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:17.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:17.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.885+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:17.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.885+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:17.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:17.889+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:17.922 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:17.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:48:17.922 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:17.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:18.097 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:18.093+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:48:18.117 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:18.113+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:18.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:18.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:18.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:18.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:18.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:18.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:18.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:18.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:18.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:18.637+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:18.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:18.637+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:18.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:18.637+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:19.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:19.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:19.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:19.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:19.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:19.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:19.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:19.521+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:19.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:19.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:19.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:19.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:19.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:19.613+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:20.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:20.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:20.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:20.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:20.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:20.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.217+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:20.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:20.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:20.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:20.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:20.733+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:22.381 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:48:22.381 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:48:22.389 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:48:22.402 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:48:22.825 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:22.821+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:48:23.631 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:48:23.631 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:48:23.632 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244503 2023-10-14T00:48:23.632 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:48:23.633 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:48:23.633 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:48:23.633 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:48:23.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.813+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:23.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:23.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:23.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:23.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:23.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:23.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.861+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:23.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:23.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:24.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:24.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:24.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:24.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:24.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:24.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:24.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:24.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:24.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:24.285+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:24.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:24.285+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:24.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:24.321+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:26.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.425+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:26.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:26.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:26.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:26.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:26.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:26.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:26.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:26.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.737+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:26.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:26.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:26.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:26.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:26.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:26.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.773+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:26.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.773+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:26.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:26.777+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:27.674 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:27.669+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:48:31.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:31.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:31.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:31.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:31.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.577+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:31.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.577+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:31.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.577+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:31.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:31.585+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:32.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:32.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:32.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:32.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:32.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:32.169+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:33.658 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:48:33.667 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:48:33.667 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:48:33.670 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:48:34.485 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:34.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:34.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.944+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:34.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:34.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:34.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:34.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:34.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:34.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.984+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:34.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.984+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:34.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:34.984+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:35.026 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:48:35.043 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:48:35.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244515 2023-10-14T00:48:35.044 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:48:35.045 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:48:35.045 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:48:35.045 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:48:37.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:37.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:37.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:37.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:37.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:37.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:37.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:37.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:37.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:37.900+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:37.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:37.900+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:37.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:37.904+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:39.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:39.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:39.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:39.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:39.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:39.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.408+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:39.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.408+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:39.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.408+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:39.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:39.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:39.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:39.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.644+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:39.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.700+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:39.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:39.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:39.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:39.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.744+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:39.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.752+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:40.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:39.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:40.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:40.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:40.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:40.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:40.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:40.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:40.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:40.028+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:40.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:40.028+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:40.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:40.064+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:41.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:41.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:41.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:41.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:41.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:41.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.208+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:41.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.216+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:41.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.868+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:41.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:41.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:41.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:41.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:41.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.892+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:41.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.892+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:41.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:41.904+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:42.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.264+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:42.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:42.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:42.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:42.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:42.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:42.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.364+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:42.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.364+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:42.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:42.364+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:43.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.360+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:43.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:43.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:43.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:43.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:43.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:43.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.412+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:43.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.412+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:43.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:43.416+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:44.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:44.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:44.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:44.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:44.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:44.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:44.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:44.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:44.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:44.860+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:45.025 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:45.020+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:48:45.025 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:45.020+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:45.067 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:48:45.067 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:48:45.084 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:48:45.095 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:48:46.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:46.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:46.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.152+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:46.411 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:48:46.420 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:48:46.421 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244526 2023-10-14T00:48:46.421 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:48:46.422 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:48:46.422 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:48:46.422 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:48:46.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.732+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:46.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:46.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:46.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:46.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:46.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:46.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.772+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:46.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.772+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:46.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:46.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.988+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:46.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:46.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:47.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:47.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:47.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:47.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:47.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:47.020+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:48.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:48.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:48.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:48.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:48.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:48.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:48.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:48.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:48.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:48.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:48.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:48.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:48.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:48.800+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:49.542 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:49.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:48:51.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:51.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:51.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:51.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:51.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:51.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:51.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:51.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:52.903 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:52.900+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:48:52.912 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:52.908+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:48:52.931 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:52.928+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:53.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.044+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:53.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:53.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:53.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:53.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:53.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.112+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:53.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.112+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:53.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:53.124+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:55.572 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:55.568+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:48:55.606 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:55.600+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:48:55.607 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:55.600+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:56.441 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:48:56.454 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:48:56.454 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:48:56.464 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:48:56.597 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:56.592+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:57.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.728+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:57.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:57.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:57.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.772+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:57.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.772+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:57.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:57.784+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:57.790 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:48:57.791 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:48:57.804 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244537 2023-10-14T00:48:57.804 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:48:57.804 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:48:57.805 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:48:57.805 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:48:58.046 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:48:58.044+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:48:59.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.236+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:59.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:59.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:59.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:59.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:59.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:59.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.268+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:59.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.280+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:48:59.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:48:59.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:48:59.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.880+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:48:59.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:48:59.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:48:59.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:48:59.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:48:59.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:48:59.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.928+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:48:59.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.928+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:48:59.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:48:59.928+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:00.178 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:00.172+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:00.194 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:00.188+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:00.494 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:00.492+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:49:00.527 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:00.524+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:00.539 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:00.536+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:01.013 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:01.008+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:01.020 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:01.016+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:02.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.276+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:02.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:02.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:02.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:02.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:02.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.344+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:02.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.344+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:02.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:02.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:03.496 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:49:03.496+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:49:04.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:04.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:04.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:04.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:04.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:04.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:04.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:04.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:04.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:04.800+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:04.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:04.800+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:06.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:06.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:06.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:06.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:06.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:06.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.412+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:06.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.412+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:06.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.628+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:06.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:06.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:06.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:06.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.672+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:06.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:06.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:06.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:06.720+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:07.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:07.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:07.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:07.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:07.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:07.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.184+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:07.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.184+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:07.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.184+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:07.712 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:07.712+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:49:07.715 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:07.716+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:49:07.718 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:07.716+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:07.742 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:07.744+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:07.843 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:49:07.846 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:49:07.854 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:49:07.858 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:49:07.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.996+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:07.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:07.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:08.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:08.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:08.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:08.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.048+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:08.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.048+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:08.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.048+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:08.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:08.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:08.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:08.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:08.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:08.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:08.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.416+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:08.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:08.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:08.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:08.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.944+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:08.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:08.944+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:09.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:09.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:09.219 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:49:09.243 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:49:09.248 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244549 2023-10-14T00:49:09.248 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:49:09.249 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:49:09.249 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:49:09.249 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:49:10.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.416+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:10.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:10.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:10.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:10.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:10.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.424+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:10.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.424+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:10.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.428+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:10.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:10.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:10.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:10.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:10.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:10.984+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:11.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.024+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:11.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.028+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:49:11.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.028+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:49:11.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.052+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:11.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:11.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:11.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:11.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:11.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:11.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.072+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:11.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:11.072+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:11.116 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:11.116+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:11.116 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:11.116+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:11.946 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:11.944+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:13.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:13.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:13.087 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:13.087+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:13.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:13.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:13.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:13.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:13.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:13.115+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:13.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:13.115+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:13.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:13.115+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:13.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:13.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:14.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:14.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:14.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:14.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:14.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:14.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.483+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:14.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.483+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:14.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:14.483+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:15.384 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:15.383+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:49:15.385 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:15.383+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:15.645 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:15.643+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:16.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:16.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:16.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:16.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:16.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:16.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:16.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:16.099+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:16.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:16.131+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:16.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:16.159+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:16.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:16.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:49:16.346 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:16.347+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:16.355 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:16.355+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:17.615 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:17.611+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:18.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:18.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:18.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:18.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:18.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:18.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:18.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:18.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:18.789 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:18.787+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:18.806 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:18.803+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:18.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:18.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:19.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:19.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:19.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:19.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:19.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:19.019+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:19.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:19.019+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:19.277 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:49:19.285 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:49:19.286 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:49:19.290 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:49:20.590 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:49:20.600 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:49:20.601 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244560 2023-10-14T00:49:20.601 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:49:20.601 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:49:20.602 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:49:20.602 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:49:21.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:21.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:21.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:21.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:21.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:21.159+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:22.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.247+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:22.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:22.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:22.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:22.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:22.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:22.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.315+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:22.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.315+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:22.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:22.315+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:23.161 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:23.159+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:23.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:23.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:23.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:23.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:23.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:23.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.647+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:23.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.647+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:23.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:23.655+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:24.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:24.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:25.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:25.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:25.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:25.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:25.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.027+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:25.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.027+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:25.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.039+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:25.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:25.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:25.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:25.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:25.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:25.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.679+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:25.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.679+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:25.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:25.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:25.714 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:25.711+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:49:25.743 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:25.743+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:26.165 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:26.163+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:26.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:26.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:26.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:26.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:26.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:26.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.683+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:26.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.703+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:26.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:26.711+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:27.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:27.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:27.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:27.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:27.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.235+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:27.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.235+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:27.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.271+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:27.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:27.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:27.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:27.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:28.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.651+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:28.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:28.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:28.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:28.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:28.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:28.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.695+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:28.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.695+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:28.719 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:28.719+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:49:28.729 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:28.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:28.729 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:28.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:28.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.727+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:28.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:28.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:28.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:28.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:29.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:29.011+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:30.612 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:49:30.617 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:49:30.619 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:49:30.637 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:49:30.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.827+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:49:30.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.827+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:49:30.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.879+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:30.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:30.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:30.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:30.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:30.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:30.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:30.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:30.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:30.919+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:31.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:31.579+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:31.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:31.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:31.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:31.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:31.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:31.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:31.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:31.615+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:31.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:31.615+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:31.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:31.619+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:31.923 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:31.923+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:49:31.952 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:31.951+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:31.963 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:31.963+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:32.025 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:49:32.025 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:49:32.026 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244572 2023-10-14T00:49:32.027 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:49:32.027 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:49:32.027 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:49:32.027 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:49:33.159 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:49:33.159+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:49:33.160 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:49:33.159+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:49:34.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:34.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:34.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:34.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.639+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:34.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.639+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:34.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.651+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:34.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:34.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:34.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:34.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:34.727+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:35.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:35.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:35.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:35.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:35.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:35.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.639+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:35.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.639+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:35.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:35.659+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:36.328 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:36.327+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:37.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:37.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:37.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:37.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:37.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:37.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.411+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:37.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.411+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:37.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:37.419+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:38.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:38.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:38.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:38.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:38.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:38.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.419+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:38.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:38.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:38.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:38.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:38.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.479+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:38.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:38.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:38.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:38.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:38.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.763+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:38.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.763+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:38.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.767+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:38.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:38.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:38.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:38.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:38.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:38.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.879+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:38.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.879+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:38.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:38.903+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:41.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.071+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:49:41.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.071+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:49:41.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.075+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:41.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:41.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:41.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:41.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:41.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:41.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.083+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:41.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.083+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:41.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.083+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:41.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.199+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:41.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:41.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:41.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:41.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:41.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:41.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.263+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:41.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.263+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:41.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:41.263+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:42.044 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:49:42.045 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:49:42.051 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:49:42.069 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:49:43.297 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:49:43.297 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:49:43.306 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244583 2023-10-14T00:49:43.306 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:49:43.306 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:49:43.306 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:49:43.307 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:49:43.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:43.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:43.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:43.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:43.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:43.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.667+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:43.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:43.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:43.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:43.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:43.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:43.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:43.735+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:44.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.023+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:44.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:44.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:44.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:44.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:44.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:44.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.043+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:44.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.043+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:44.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:44.043+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:45.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.767+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:45.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:45.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:45.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:45.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:45.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:45.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.811+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:45.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.811+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:45.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.811+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:45.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:45.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:45.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:46.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.395+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:46.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.563+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:46.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:46.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:46.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:46.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:46.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:46.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.695+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:46.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.695+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:46.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:46.695+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:47.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:47.707+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:47.717 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:47.715+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:47.729 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:47.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:49:47.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:47.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:47.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:47.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:47.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:47.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:47.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:47.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:47.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:47.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:48.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.507+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:48.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:48.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:48.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:48.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:48.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.551+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:48.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.551+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:48.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.551+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:48.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.563+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:48.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:48.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:48.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:49.456 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:49.455+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:49:49.462 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:49.463+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:49:49.468 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:49.467+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:49:49.480 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:49:49.479+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:49:50.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:50.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:50.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:50.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:50.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:50.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:50.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:50.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:50.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:50.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:50.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:50.851+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:51.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.663+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:51.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:51.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:49:51.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:51.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:51.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:51.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.731+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:51.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.731+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:51.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:51.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:53.319 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:49:53.319 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:49:53.319 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:49:53.320 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:49:54.568 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:49:54.568 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:49:54.569 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244594 2023-10-14T00:49:54.569 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:49:54.569 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:49:54.570 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:49:54.570 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:49:56.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:56.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:49:58.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.442+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:49:58.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:49:58.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:49:58.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:49:58.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:49:58.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.486+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:49:58.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.486+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:49:58.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:49:58.486+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:00.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:00.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:00.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:00.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:00.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:00.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:00.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:00.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:00.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:00.674+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:01.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.290+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:01.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:01.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:01.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:01.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:01.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:01.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.430+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:01.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.430+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:01.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.438+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:01.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.542+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:01.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:01.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:01.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:01.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:01.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:01.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:01.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.946+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:01.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.946+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:01.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:01.946+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:04.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.130+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:04.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:04.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:04.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:04.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:04.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:04.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.150+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:04.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.150+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:04.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.158+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:04.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:04.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:04.587 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:50:04.604 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:50:04.606 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:50:04.607 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:50:05.262 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:05.262+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:06.022 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:06.022+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:50:06.043 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:06.042+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:06.119 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:50:06.127 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:50:06.131 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244606 2023-10-14T00:50:06.131 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:50:06.131 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:50:06.132 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:50:06.132 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:50:06.677 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:06.674+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:50:06.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.818+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:06.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:06.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:06.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:06.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:06.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.834+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:06.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.834+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:06.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:06.834+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:07.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.414+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:07.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:07.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:07.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:07.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:07.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:07.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.462+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:07.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.462+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:07.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.462+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:07.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:07.774+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:08.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:08.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:08.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:08.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:08.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:08.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.338+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:08.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.350+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:08.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.362+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:50:08.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:08.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:08.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:09.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.382+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:09.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:09.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.398+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:09.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.814+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:09.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:09.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:09.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:09.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:09.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.838+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:09.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:09.838+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:10.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:10.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:10.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:10.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:10.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:10.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.942+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:10.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.946+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:10.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.946+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:10.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.958+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:50:10.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.958+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:50:10.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.974+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:10.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:10.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:11.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:10.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:11.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:11.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:11.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:11.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:11.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:11.014+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:11.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:11.014+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:11.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:11.014+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:11.098 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:11.094+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:50:11.111 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:11.110+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:50:11.142 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:11.138+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:12.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:12.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:12.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:12.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:12.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:12.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:12.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:12.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:12.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:12.730+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:12.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:12.730+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:12.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:12.738+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:12.819 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:12.818+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:50:12.834 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:12.834+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:15.766 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:15.762+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:50:16.134 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:50:16.134 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:50:16.141 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:50:16.149 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:50:16.231 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:16.230+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:50:16.231 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:16.230+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:16.266 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:16.262+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:50:16.346 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:16.346+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:17.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:17.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:17.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:17.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:17.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.374+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:17.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.374+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:17.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.410+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:17.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:17.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:17.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:17.502 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:50:17.506 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:50:17.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:17.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:17.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.506+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:17.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.506+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:17.517 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244617 2023-10-14T00:50:17.517 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:50:17.518 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:50:17.518 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:50:17.518 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:50:17.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:17.538+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:18.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:18.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:18.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:18.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:18.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:18.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:18.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:18.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:18.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:18.494+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:18.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:18.494+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:18.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:18.498+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:19.216 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:19.214+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:50:19.265 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:19.262+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:50:19.284 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:19.282+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:20.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:20.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:20.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:20.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:20.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:20.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:20.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:20.154+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:22.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.014+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:50:22.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.014+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:50:22.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.038+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:22.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:22.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:22.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:22.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:22.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:22.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.074+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:22.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:22.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:22.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:50:22.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:50:22.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.130+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:22.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:22.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:22.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:22.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:22.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:22.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.162+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:22.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:22.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:22.220 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:22.218+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:22.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:22.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:22.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:22.498+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:24.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:24.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:24.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:24.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:24.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:24.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:24.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:24.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:24.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:25.152 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:25.146+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:27.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:27.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:27.525 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:50:27.531 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:50:27.534 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:50:27.546 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:50:28.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:28.514+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:28.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:28.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:28.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:28.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:28.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:28.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:28.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:28.534+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:28.932 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:50:28.932 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:50:28.933 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244628 2023-10-14T00:50:28.933 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:50:28.933 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:50:28.933 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:50:28.934 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:50:29.731 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:29.730+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:30.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.070+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:30.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:30.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:30.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:30.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:30.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:30.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.094+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:30.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.094+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:30.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.094+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:30.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.546+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:30.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:30.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:30.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:30.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:30.566+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:32.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:32.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:32.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:32.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:32.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:32.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:32.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:32.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:32.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:32.326+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:33.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:33.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:33.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:33.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:33.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:33.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:33.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:33.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:33.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:33.597+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:33.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:33.597+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:33.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:33.597+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:34.431 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:50:34.429+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:50:34.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:34.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:34.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:34.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:34.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:34.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:35.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:35.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:35.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:35.017+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:35.603 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:35.601+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:37.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.869+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:37.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:37.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:37.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:37.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:37.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:37.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.925+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:37.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.925+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:37.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:37.925+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:38.939 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:50:38.940 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:50:38.944 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:50:38.966 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:50:38.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:38.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:39.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:39.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:39.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:39.069+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:50:39.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:39.069+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:50:39.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:39.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:39.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:39.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:39.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:39.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:40.296 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:50:40.297 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:50:40.297 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244640 2023-10-14T00:50:40.298 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:50:40.298 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:50:40.298 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:50:40.298 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:50:40.731 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:40.729+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:50:40.756 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:40.753+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:50:40.758 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:40.757+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:41.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:41.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:41.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.745+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:41.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:41.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:41.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.765+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:41.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.765+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:41.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:41.765+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:44.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:44.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:44.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:44.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:44.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:44.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:44.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:44.373+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:44.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:44.373+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:44.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:44.373+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:44.723 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:50:44.721+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:50:45.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:45.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:45.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:45.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:45.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:45.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:45.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:45.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:45.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:45.129+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:45.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:45.129+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:45.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:45.129+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:49.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:49.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:49.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:49.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:49.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:49.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:49.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:49.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:49.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:49.361+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:49.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:49.361+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:49.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:49.373+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:50.302 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:50:50.307 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:50:50.310 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:50:50.314 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:50:51.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.261+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:51.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:51.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:51.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:51.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:51.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:51.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.313+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:51.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.329+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:51.549 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:50:51.556 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:50:51.557 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244651 2023-10-14T00:50:51.557 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:50:51.557 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:50:51.557 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:50:51.558 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:50:51.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:51.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:51.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:51.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:51.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:51.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.701+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:51.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.701+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:51.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:51.705+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:53.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:53.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:53.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:53.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:53.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:53.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:53.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:53.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:53.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:53.857+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:53.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:53.889+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:53.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:53.897+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:55.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:55.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:55.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:55.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:55.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:55.837+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:58.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:58.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:58.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:58.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.173+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:50:58.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.261+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:50:58.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:50:58.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:50:58.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:50:58.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:50:58.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:50:58.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:50:58.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:50:58.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:50:58.409+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:00.677 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:00.673+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:00.699 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:00.697+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:01.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.513+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:01.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:01.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:01.586 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:51:01.588 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:51:01.598 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:51:01.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:01.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:01.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.593+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:01.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.593+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:01.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:01.605+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:01.611 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:51:02.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.425+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:02.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:02.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:02.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:02.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:02.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.445+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:02.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.445+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:02.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:02.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:02.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:02.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:02.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:02.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:02.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.737+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:02.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:02.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:03.114 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:51:03.114 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:51:03.114 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244663 2023-10-14T00:51:03.115 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:51:03.115 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:51:03.115 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:51:03.115 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:51:04.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:04.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:04.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:04.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:04.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:04.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.121+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:04.426 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:04.417+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:04.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:04.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:04.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:04.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:04.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.925+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:04.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.925+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:04.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:04.929+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:05.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:05.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:05.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:05.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:05.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:05.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.105+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:05.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.105+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:05.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:05.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:06.635 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:06.625+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:06.657 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:06.653+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:08.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:08.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:10.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:10.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:10.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:10.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:10.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:10.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:11.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:11.009+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:11.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:11.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:11.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:11.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:11.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:11.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:11.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:11.581+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:12.112 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:12.109+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:13.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:13.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:13.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.081+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:13.127 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:51:13.133 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:51:13.138 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:51:13.146 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:51:13.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.445+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:13.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:13.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:13.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:13.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.489+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:13.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:13.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:13.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:13.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:13.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.817+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:13.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.821+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:13.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:13.893+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:14.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:14.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:14.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:14.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:14.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:14.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.225+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:14.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:14.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:14.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:14.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.265+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.265+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:14.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:14.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:14.341 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:51:14.342 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:51:14.342 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244674 2023-10-14T00:51:14.342 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:51:14.342 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:51:14.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:51:14.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:51:18.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:18.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:18.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:18.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:18.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:18.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:18.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:18.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:19.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:19.008+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:20.064 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:20.060+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:20.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.412+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:20.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:20.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:20.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:20.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:20.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.444+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:20.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.452+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:20.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:20.456+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:22.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.632+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:22.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:22.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:22.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:22.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:22.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:22.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.696+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:22.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.696+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:22.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:22.700+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:23.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:23.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:23.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:23.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:23.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:23.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:23.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:23.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:23.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:23.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:23.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:23.800+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:23.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:23.800+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:24.347 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:51:24.348 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:51:24.352 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:51:24.367 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:51:25.506 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:51:25.508 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:51:25.514 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244685 2023-10-14T00:51:25.514 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:51:25.515 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:51:25.515 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:51:25.515 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:51:25.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:25.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:25.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:25.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:25.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:25.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:25.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:25.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:25.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:25.740+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:25.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:25.740+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:25.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:25.752+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:27.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:27.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:27.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:27.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:27.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:27.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:27.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:27.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:27.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:27.072+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:27.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:27.072+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:27.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:27.072+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:27.638 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:27.636+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:51:27.640 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:27.636+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:28.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.284+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:28.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:28.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:28.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:28.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:28.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:28.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:28.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.340+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:28.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.352+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:28.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:28.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:28.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:28.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:28.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:28.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:28.760+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:32.826 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:32.820+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:51:32.836 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:32.832+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:32.861 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:32.856+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:33.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.216+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:33.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:33.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:33.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:33.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:33.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:33.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.300+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:33.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.300+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:33.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:33.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:33.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:33.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:33.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:33.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:33.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.924+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:33.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.924+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:33.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:33.948+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:35.518 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:51:35.522 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:51:35.533 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:51:35.545 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:51:36.969 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:51:36.970 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:51:36.971 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244696 2023-10-14T00:51:36.971 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:51:36.971 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:51:36.971 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:51:36.972 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:51:37.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:37.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:37.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:37.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:37.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:37.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:37.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:37.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:37.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:37.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:37.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:37.324+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:37.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:37.336+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:38.571 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:38.568+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:51:38.610 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:38.608+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:38.623 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:38.620+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:38.650 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:38.648+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:51:38.678 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:38.672+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:51:38.694 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:38.688+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:38.737 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:38.732+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:39.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.640+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:39.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:39.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:39.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:39.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:39.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.656+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:39.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.656+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:39.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:39.656+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:40.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.592+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:40.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:40.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:40.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:40.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:40.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:40.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.660+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:40.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.660+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:40.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:40.660+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:41.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:41.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:41.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:41.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:41.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:41.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.140+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:41.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.140+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:41.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.140+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:41.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:41.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:41.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:41.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:41.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.692+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:41.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.692+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:41.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:41.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:42.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:42.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:42.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:42.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:42.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:42.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:42.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:42.044+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:43.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:43.988+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:43.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:43.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:44.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:44.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:44.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:44.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.008+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:44.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.008+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:44.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.008+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:44.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:44.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:44.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:44.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:44.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.528+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:44.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:44.536+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:45.722 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:45.716+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:51:45.740 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:45.736+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:45.748 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:45.744+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:46.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:46.944+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:46.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:46.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:46.980 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:51:46.982 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:51:46.982 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:51:46.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:47.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:47.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:47.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:47.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:47.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:47.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:47.000+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:47.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:47.000+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:47.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:47.000+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:47.002 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:51:48.469 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:51:48.470 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:51:48.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244708 2023-10-14T00:51:48.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:51:48.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:51:48.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:51:48.472 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:51:50.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:50.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:50.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:50.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:50.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:50.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:50.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:50.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:50.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:50.376+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:52.544 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:52.544+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:52.578 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:52.576+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:51:52.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.856+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:52.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:52.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:52.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:52.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:52.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:52.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.904+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:52.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.904+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:52.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:52.908+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:53.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.768+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:51:53.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.768+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:51:53.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.768+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:53.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:53.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:53.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:53.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:53.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:53.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:53.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.800+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:53.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:53.800+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:54.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:54.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:54.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:54.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:54.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:54.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:54.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:54.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:54.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:54.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:54.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:54.624+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:54.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:54.644+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:55.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:55.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:55.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:55.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:55.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:55.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:55.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:55.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:55.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:55.168+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:55.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:55.168+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:55.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:55.208+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:55.496 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:51:55.496+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:51:57.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:57.563+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:51:57.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:57.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:57.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:57.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:57.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:57.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:57.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:57.575+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:57.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:57.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:58.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:51:58.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:51:58.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:51:58.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:51:58.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:51:58.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.243+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:51:58.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.243+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:51:58.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:51:58.259+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:51:58.474 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:51:58.475 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:51:58.475 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:51:58.476 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:51:59.723 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:51:59.729 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:51:59.730 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244719 2023-10-14T00:51:59.730 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:51:59.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:51:59.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:51:59.731 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:52:00.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:00.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:00.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:00.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:00.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:00.675+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:00.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:00.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:00.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:00.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:01.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.511+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:01.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:01.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.515+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:01.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:01.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:01.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:01.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.535+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:01.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.535+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:01.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:01.535+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:02.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:02.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:02.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:02.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:02.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:02.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:02.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:02.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:02.923+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:03.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.599+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:03.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:03.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:03.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:03.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:03.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.719+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:03.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.719+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:03.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:03.723+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:05.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:05.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:05.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:05.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:05.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:05.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:05.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:05.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:05.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:05.895+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:05.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:05.895+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:05.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:05.903+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:06.435 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:06.435+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:52:06.450 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:06.451+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:07.242 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:07.239+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:09.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:09.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:09.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:09.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:09.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:09.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:09.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:09.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:09.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:09.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:09.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:09.563+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:09.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:09.591+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:09.739 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:52:09.740 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:52:09.743 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:52:09.744 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:52:11.054 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:52:11.064 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:52:11.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244731 2023-10-14T00:52:11.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:52:11.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:52:11.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:52:11.066 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:52:11.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:11.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:11.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:11.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:11.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:11.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:11.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:11.803+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:11.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:11.803+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:11.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:11.803+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:12.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:12.587+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:12.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:12.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:12.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:12.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:12.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:12.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:12.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:12.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:12.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:12.623+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:12.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:12.631+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:13.851 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:13.851+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:13.859 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:13.859+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:14.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:14.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:14.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:14.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:14.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:14.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.599+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:14.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.603+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:14.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:14.647+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:15.972 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:15.971+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:16.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.143+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:16.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:16.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:16.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:16.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:16.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:16.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.231+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:16.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.231+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:16.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:16.239+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:16.534 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:16.535+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:52:16.583 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:16.583+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:16.611 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:16.607+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:17.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.783+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:52:17.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:17.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:17.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:17.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:17.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:17.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:17.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.891+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:17.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.899+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:17.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:17.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:18.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:18.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:18.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:18.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:18.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:18.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:18.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:18.015+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:18.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:18.015+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:18.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:18.027+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:18.189 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:18.187+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:52:18.190 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:18.187+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:20.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:20.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:20.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:20.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:20.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:20.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:20.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:20.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:20.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:20.855+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:20.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:20.883+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:21.068 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:52:21.068 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:52:21.068 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:52:21.082 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:52:21.224 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:21.223+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:52:21.224 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:21.223+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:22.418 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:52:22.422 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:52:22.434 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244742 2023-10-14T00:52:22.434 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:52:22.435 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:52:22.435 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:52:22.435 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:52:23.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:23.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:23.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:23.699+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:24.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.423+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:52:24.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.435+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:24.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:24.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:24.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:24.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:24.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:24.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.483+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:24.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.483+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:24.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:24.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:25.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.155+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:25.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:25.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:25.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:25.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:25.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:25.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.255+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:25.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.255+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:25.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:25.271+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:25.528 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:25.527+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:26.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:26.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:26.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:26.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:26.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.227+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:26.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.227+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:26.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.231+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:26.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:26.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:26.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:26.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:26.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:26.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.455+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:26.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.455+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:26.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:26.455+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:26.960 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:26.959+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:27.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:27.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:27.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:27.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:27.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.119+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:27.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.147+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:52:27.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.147+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:52:27.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:27.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:27.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:27.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:27.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:27.187+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:28.323 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:28.323+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:29.101 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:29.099+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:29.134 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:29.135+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:29.311 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:29.311+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:29.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:29.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:29.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:29.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:29.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:29.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.651+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:29.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.651+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:29.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:29.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:32.357 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:32.355+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:32.440 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:52:32.441 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:52:32.441 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:52:32.441 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:52:32.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.515+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:32.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:32.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:32.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:32.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:32.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.567+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:32.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.655+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:32.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:32.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:32.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:32.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:32.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.731+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:32.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.731+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:32.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:32.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:33.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.099+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:33.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:33.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:33.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:33.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:33.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.115+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:33.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.115+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:33.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.115+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:33.745 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:52:33.745 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:52:33.746 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244753 2023-10-14T00:52:33.746 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:52:33.746 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:52:33.746 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:52:33.747 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:52:33.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.855+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:33.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:33.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:33.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:33.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:33.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:33.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:33.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:33.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:33.927+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:36.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:36.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:37.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:37.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:37.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:37.931+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:40.708 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:40.706+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:40.739 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:40.738+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:42.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.086+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:42.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:42.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:42.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:42.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:42.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.142+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:42.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.142+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:42.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:42.146+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:43.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:43.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:43.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.298+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:43.748 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:52:43.749 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:52:43.749 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:52:43.750 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:52:43.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.810+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:43.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:43.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:43.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:43.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:43.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.850+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:43.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.850+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:43.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.850+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:43.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.954+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:43.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:43.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:43.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:43.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:43.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.986+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:43.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.986+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:43.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:43.986+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:45.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.250+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:45.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:45.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:45.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:45.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:45.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:45.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.326+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:45.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.326+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:45.341 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:52:45.342 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:52:45.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244765 2023-10-14T00:52:45.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:52:45.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:52:45.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:52:45.343 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:52:45.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.342+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:45.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.398+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:45.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:45.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:45.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:45.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:45.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:45.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.442+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:45.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.442+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:45.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.450+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:45.496 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:45.494+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:45.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.710+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:45.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:45.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:45.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:45.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:45.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:45.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.742+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:45.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.742+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:45.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.758+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:45.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.982+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:45.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:45.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:45.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:45.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:46.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:46.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:46.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:46.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:46.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:46.002+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:46.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:46.002+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:46.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:46.022+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:49.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.026+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:49.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:49.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:49.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:49.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:49.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.074+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:49.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.074+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:49.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:49.474+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:53.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:53.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:53.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:53.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:53.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.102+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:53.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.102+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:53.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.102+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:53.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:53.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:53.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:53.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.266+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:53.536 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:53.534+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:53.558 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:53.554+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:53.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.630+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:53.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:53.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:53.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:53.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:53.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:53.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.670+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:53.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.670+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:53.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:53.686+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:55.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.138+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:52:55.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.138+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:52:55.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.146+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:55.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:55.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:55.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:55.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:55.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:55.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.166+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:55.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.166+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:55.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.166+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:55.357 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:52:55.369 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:52:55.373 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:52:55.376 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:52:55.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.378+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:55.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:55.405 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:55.398+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:52:55.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:55.420 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:55.418+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:52:55.420 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:52:55.418+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:52:55.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:55.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:55.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:55.458+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:56.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.362+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:52:56.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:56.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:56.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:56.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:56.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:56.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.414+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:56.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.414+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:56.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.414+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:56.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:56.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:56.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:56.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:56.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:56.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.654+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:56.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:56.686+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:56.843 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:52:56.853 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:52:56.861 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244776 2023-10-14T00:52:56.862 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:52:56.862 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:52:56.862 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:52:56.862 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:52:57.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:57.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:57.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:57.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:57.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:57.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:57.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:57.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:57.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:57.618+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:57.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:57.618+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:57.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:57.622+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:52:58.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:52:58.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:52:58.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:52:58.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:52:58.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:52:58.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.758+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:52:58.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.758+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:52:58.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:52:58.766+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:00.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:00.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:00.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:00.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.038+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:00.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:00.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:00.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.190+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:00.563 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:00.562+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:53:00.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:00.586 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:00.582+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:00.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:00.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:00.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:00.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.610+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:00.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.622+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:00.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:00.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:00.862+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:01.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:01.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:01.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:01.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:01.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:01.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:01.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:01.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:01.294+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:03.621 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:03.618+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:03.631 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:03.630+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:04.925 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:04.922+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:53:04.952 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:04.950+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:06.090 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:06.086+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:06.831 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:06.830+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:06.864 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:53:06.875 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:53:06.878 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:53:06.904 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:53:07.077 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:07.074+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:07.092 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:07.090+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:07.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:07.318+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:08.273 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:53:08.273 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:53:08.275 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244788 2023-10-14T00:53:08.275 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:53:08.275 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:53:08.276 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:53:08.276 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:53:08.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:08.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:08.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:08.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:08.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.682+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:08.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.682+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:08.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.702+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:08.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:08.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.850+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:08.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:08.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:08.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:08.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:08.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:08.866+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:09.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:09.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:09.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:09.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:09.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.430+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:09.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.434+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:09.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.654+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:09.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:09.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:09.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:09.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:09.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:09.726+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:10.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.466+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:53:10.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.466+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:53:10.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.482+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:10.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:10.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:10.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:10.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:10.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:10.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.498+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:10.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.498+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:10.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:10.502+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:11.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.298+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:11.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:11.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:11.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:11.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.318+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:11.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.338+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:11.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.554+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:11.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:11.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:11.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:11.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:11.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:11.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.606+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:11.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.606+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:11.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:11.614+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:12.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.042+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:12.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:12.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:12.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:12.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:12.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.054+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:12.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.070+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:12.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.074+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:12.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:12.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:12.538+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:12.867 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:12.866+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:12.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:12.870+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:13.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:13.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:13.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:13.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:13.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:13.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:13.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:13.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:13.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:13.830+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:13.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:13.830+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:15.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:15.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:15.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:15.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:17.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.218+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:17.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:17.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:17.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:17.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:17.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.262+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:17.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:17.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:17.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:17.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:17.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.618+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:17.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.618+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:17.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:17.638+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:18.278 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:53:18.279 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:53:18.279 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:53:18.285 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:53:19.423 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:19.414+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:19.892 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:53:19.900 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:53:19.903 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244799 2023-10-14T00:53:19.903 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:53:19.904 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:53:19.904 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:53:19.904 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:53:20.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:20.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:20.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:20.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.358+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:20.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.358+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:20.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.722+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:53:20.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.730+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:20.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:20.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.762+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:20.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:20.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:20.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:20.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.766+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:20.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.766+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:20.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:20.778+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:22.399 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:22.398+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:25.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:25.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:25.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:25.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:25.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:25.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:25.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:25.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:25.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:25.669+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:25.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:25.669+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:25.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:25.681+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:26.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:26.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:26.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:26.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:26.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:26.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.457+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:26.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.477+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:26.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:26.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:27.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:27.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:27.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:27.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:27.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:27.741+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:27.867 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:27.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:27.878 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:27.877+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:29.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:29.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:29.904 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:53:29.906 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:53:29.909 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:53:29.910 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:53:29.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:29.941+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:29.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:29.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:29.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:29.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:29.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:29.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:29.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:29.953+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:30.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:30.017+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:30.478 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:30.473+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:30.489 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:30.485+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:31.424 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:53:31.425 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:53:31.425 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244811 2023-10-14T00:53:31.426 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:53:31.426 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:53:31.426 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:53:31.426 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:53:31.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:31.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:31.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:31.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:31.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:31.517+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:32.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:32.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:32.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:32.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:32.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:32.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:32.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:32.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:32.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:32.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:32.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:32.317+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:32.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:32.337+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:33.627 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:33.625+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:53:33.653 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:33.649+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:33.653 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:33.649+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:33.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:33.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:33.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:33.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:33.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:33.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:33.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:33.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:33.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:33.753+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:33.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:33.753+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:33.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:33.753+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:35.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:35.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:35.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:35.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:35.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:35.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:35.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:35.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:35.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:35.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:35.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:35.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:35.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:35.401+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:36.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:36.209+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:36.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:36.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:36.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:36.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:36.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:36.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:36.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:36.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:36.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:36.297+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:37.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:37.961+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:37.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:37.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:37.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:37.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:37.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:37.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:37.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:37.981+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:39.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:39.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:39.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:39.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:39.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:39.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:39.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:39.221+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:39.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:39.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:39.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:39.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:39.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:39.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:40.688 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:40.685+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:53:40.691 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:40.689+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:40.691 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:40.689+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:41.179 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:41.173+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:53:41.188 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:41.185+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:41.209 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:41.209+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:41.401 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:41.397+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:41.459 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:53:41.478 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:53:41.489 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:53:41.493 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:53:41.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:41.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:41.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:41.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:41.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:41.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.717+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:41.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.717+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:41.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:41.717+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:42.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:42.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:42.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:42.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:42.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:42.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:42.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:42.093+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:42.608 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:53:42.608 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:53:42.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244822 2023-10-14T00:53:42.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:53:42.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:53:42.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:53:42.612 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:53:43.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:43.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:43.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:43.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:43.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:43.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:43.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:43.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:43.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:43.801+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:45.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:45.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:45.692 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:45.689+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:45.693 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:45.689+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:46.524 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:46.521+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:46.532 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:46.525+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:47.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:47.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:47.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:47.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:47.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:47.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:47.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:47.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:49.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:49.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:49.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:49.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:49.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:49.213+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:52.614 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:53:52.622 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:53:52.627 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:53:52.636 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:53:53.952 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:53:53.953 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:53:53.953 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244833 2023-10-14T00:53:53.954 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:53:53.954 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:53:53.954 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:53:53.954 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:53:54.093 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:54.089+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:53:55.382 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:55.377+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:53:56.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.473+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:56.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:56.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:56.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:56.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:56.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:56.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.545+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:56.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.545+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:56.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:56.545+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:58.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.405+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:53:58.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:53:58.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:58.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:58.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:58.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:58.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.457+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:53:58.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.457+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:53:58.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:58.477+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:59.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:59.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:53:59.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:59.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:53:59.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:59.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:53:59.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:59.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:53:59.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:53:59.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:53:59.990 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:53:59.989+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:00.058 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:00.057+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:00.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:00.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:01.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:01.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:01.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:01.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:01.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:01.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:01.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:01.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:01.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:01.077+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:02.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:02.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:02.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:02.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:03.800 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:03.797+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:54:03.854 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:03.853+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:03.961 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:54:03.962 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:54:03.963 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:54:03.984 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:54:04.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.201+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:04.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:04.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:04.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:04.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:04.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:04.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.265+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:04.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.265+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:04.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:04.281+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:04.307 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:04.305+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:54:04.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:04.321+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:04.334 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:04.333+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:05.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:05.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:05.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:05.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:05.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:05.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:05.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:05.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:05.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:05.017+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:05.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:05.017+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:05.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:05.017+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:05.287 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:54:05.293 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:54:05.295 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244845 2023-10-14T00:54:05.295 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:54:05.295 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:54:05.296 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:54:05.296 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:54:06.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:06.617+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:06.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:06.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:06.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:06.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:06.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:06.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:06.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:06.657+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:07.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.649+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:54:07.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.677+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:54:07.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.681+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:07.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:07.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:07.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:07.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:07.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:07.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.761+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:07.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.761+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:07.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:07.769+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:09.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.524+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:09.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:09.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:09.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:09.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:09.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.580+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:09.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.580+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:09.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:09.588+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:10.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.496+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:10.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:10.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:10.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:10.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.536+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:10.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.536+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:10.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.536+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:10.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.904+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:54:10.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.904+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:54:10.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.908+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:10.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:10.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:10.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:10.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:10.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:10.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.952+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:10.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.952+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:10.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:10.960+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:11.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:11.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:11.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:11.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.152+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:11.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.152+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:11.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:11.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:11.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:11.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:11.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:11.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.316+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:11.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:11.316+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:11.353 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:11.348+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:11.359 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:11.356+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:11.746 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:11.744+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:12.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:12.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:12.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:12.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:12.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.432+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:12.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.432+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:12.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.432+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:12.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:12.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:12.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:12.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:12.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.732+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:12.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.732+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:12.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.732+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:12.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:12.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:12.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:12.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:12.808+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:14.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:14.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:14.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.304+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:14.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:14.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:14.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:14.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.824+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:14.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:14.844+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:15.299 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:54:15.300 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:54:15.303 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:54:15.304 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:54:15.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:15.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:15.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:15.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:15.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:15.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:15.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:15.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:15.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:15.636+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:15.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:15.636+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:15.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:15.644+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:15.697 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:15.692+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:16.709 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:54:16.710 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:54:16.711 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244856 2023-10-14T00:54:16.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:54:16.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:54:16.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:54:16.712 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:54:16.763 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:16.756+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:16.765 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:16.760+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:16.809 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:16.804+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:54:16.816 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:16.812+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:16.838 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:16.836+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:17.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:17.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:17.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:17.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:17.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:17.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:17.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:17.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:17.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:20.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.328+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:54:20.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.328+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:54:20.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.348+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:20.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:20.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:20.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:20.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:20.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:20.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.388+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:20.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.388+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:20.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:20.392+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:22.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.460+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:22.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:22.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:22.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:22.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:22.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:22.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.500+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:22.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:22.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:23.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:23.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:23.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:23.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:23.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:23.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.224+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:23.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.224+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:23.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.264+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:23.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.276+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:23.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:23.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:23.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:23.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:23.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:23.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:23.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:23.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:23.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:23.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.884+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:23.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.884+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:23.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:23.904+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:23.954 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:54:23.952+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:54:23.954 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:54:23.952+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:54:24.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.252+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:24.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:24.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:24.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:24.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:24.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:24.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.276+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:24.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.276+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:24.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:24.276+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:25.369 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:25.364+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:26.716 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:54:26.716 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:54:26.721 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:54:26.743 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:54:26.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:26.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:26.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:26.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:26.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:26.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:26.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:26.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:26.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:26.936+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:26.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:26.944+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:26.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:26.976+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:27.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:27.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:27.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:27.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:27.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:27.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:27.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:27.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:27.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:27.504+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:27.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:27.504+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:27.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:27.556+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:28.184 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:54:28.203 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:54:28.211 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244868 2023-10-14T00:54:28.211 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:54:28.211 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:54:28.212 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:54:28.212 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:54:28.534 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:28.524+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:28.551 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:28.548+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:28.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:28.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:28.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:28.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:28.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:28.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.796+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:28.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.796+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:28.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:28.796+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:29.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:29.536+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:29.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:29.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:29.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:29.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:29.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:29.708+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:32.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.300+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:54:32.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.300+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:54:32.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.300+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:32.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:32.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:32.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:32.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:32.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:32.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.348+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:32.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.348+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:32.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.360+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:32.367 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:32.364+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:54:32.375 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:32.372+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:54:32.377 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:32.376+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:32.381 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:32.376+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:32.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.376+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:32.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:32.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:32.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:32.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:32.412+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:33.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.324+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:33.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:33.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:33.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.352+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:33.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.408+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:33.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:33.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:33.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:33.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:33.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:33.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:33.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:33.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:33.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:33.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:33.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:33.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:33.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:35.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:35.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:35.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:35.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.024+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:35.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.024+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:35.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.092+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:35.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:35.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:35.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:35.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:35.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:35.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.164+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:35.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.164+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:35.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:35.172+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:36.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:36.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:36.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:36.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:36.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.204+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:36.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.208+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:54:36.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.208+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:54:36.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.208+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:36.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:36.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:36.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:36.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:36.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:36.260+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:37.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.612+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:37.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:37.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:37.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:37.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:37.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:37.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.700+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:37.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.700+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:37.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:37.724+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:38.219 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:54:38.220 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:54:38.220 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:54:38.237 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:54:38.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.368+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:38.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:38.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:38.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:38.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:38.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:38.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:38.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:38.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:38.448+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:39.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:39.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:39.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:39.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:39.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:39.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.084+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:39.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.084+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:39.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:39.088+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:39.748 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:54:39.748 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:54:39.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244879 2023-10-14T00:54:39.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:54:39.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:54:39.749 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:54:39.750 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:54:40.862 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:40.864+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:41.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:41.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:41.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:41.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:41.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:41.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:41.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:41.724+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:41.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:41.724+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:41.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:41.724+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:41.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:41.732+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:42.709 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:42.708+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:54:42.734 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:42.736+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:43.173 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:43.172+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:44.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:44.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:44.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:44.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:44.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:44.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.520+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:44.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.520+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:44.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:44.528+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:46.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.724+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:46.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:46.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:46.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:46.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:46.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.779+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:46.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.779+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:46.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:46.779+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:47.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:47.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:47.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.311+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:47.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.671+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:54:47.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.671+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:54:47.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.671+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:47.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:47.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:47.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:47.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:47.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:47.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.707+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:47.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.707+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:47.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:47.711+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:48.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:48.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:48.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:48.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:48.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:48.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:48.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:48.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:48.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:48.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:48.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:48.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:48.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:48.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:49.756 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:54:49.757 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:54:49.765 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:54:49.765 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:54:49.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:49.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:49.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:49.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:49.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:49.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:49.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:49.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:49.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:49.883+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:49.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:49.883+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:49.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:49.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:50.964 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:54:50.966 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:54:50.967 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244890 2023-10-14T00:54:50.967 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:54:50.967 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:54:50.967 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:54:50.968 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:54:51.835 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:51.835+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:51.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:51.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:51.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:51.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:51.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:51.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:51.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:51.875+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:51.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:51.875+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:51.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:51.883+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:52.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.047+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:52.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:52.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:52.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:52.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:52.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.079+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:52.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.079+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:52.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.079+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:52.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:52.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.819+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:52.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:52.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:52.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:52.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:52.839+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:53.470 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:53.471+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:53.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:53.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:53.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:53.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:53.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:53.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:53.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:53.483+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:53.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:53.483+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:53.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:53.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:54.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:54.391+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:54:54.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:54.403+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:54:54.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:54.403+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:54.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:54.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:54.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:54.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:54.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:54.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:54.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:54.459+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:55.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:55.847+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:55.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:55.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:55.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:55.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:55.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:55.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:55.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:55.923+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:56.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:56.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:56.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:56.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:56.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:56.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:56.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:56.499+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:57.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:57.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:57.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:57.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:57.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:57.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.327+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:57.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.495+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:57.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:54:57.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.515+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:57.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:57.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:57.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.535+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:57.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.535+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:57.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:57.535+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:58.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.267+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:54:58.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:54:58.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:54:58.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:54:58.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:54:58.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.379+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:54:58.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.379+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:54:58.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:58.383+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:54:58.698 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:58.699+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:59.148 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:59.147+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:54:59.148 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:59.147+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:54:59.148 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:54:59.147+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:54:59.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:54:59.991+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:00.969 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:55:00.969 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:55:00.969 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:55:00.980 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:55:01.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.355+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:01.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:01.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:01.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:01.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:01.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:01.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.487+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:01.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.487+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:01.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:01.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:01.522 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:01.519+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:01.559 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:01.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:02.370 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:55:02.371 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:55:02.372 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244902 2023-10-14T00:55:02.372 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:55:02.372 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:55:02.372 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:55:02.372 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:55:03.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:03.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:03.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:03.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:03.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:03.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:03.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:03.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:03.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:03.295+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:03.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:03.303+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:03.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:03.303+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:07.831 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:07.831+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:07.841 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:07.839+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:10.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.323+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:10.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:10.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:10.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:10.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:10.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.375+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:10.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.375+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:10.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:10.375+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:11.411 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:11.411+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:55:11.431 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:11.431+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:12.374 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:55:12.374 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:55:12.375 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:55:12.376 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:55:12.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:12.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:12.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:12.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:12.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:12.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:12.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:12.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:12.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:12.875+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:12.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:12.875+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:12.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:12.875+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:13.488 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:55:13.489 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:55:13.489 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244913 2023-10-14T00:55:13.489 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:55:13.490 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:55:13.490 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:55:13.490 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:55:13.844 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:13.843+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:15.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.851+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:55:15.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.863+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:55:15.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.863+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:15.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:15.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:15.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:15.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:15.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:15.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:15.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:15.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:16.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.003+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:55:16.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.027+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:16.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:16.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:16.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:16.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:16.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:16.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.063+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:16.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.063+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:16.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:16.091+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:20.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:20.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:20.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:20.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.195+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:20.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.195+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:20.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.195+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:20.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.479+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:20.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:20.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:20.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:20.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:20.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.531+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:20.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.531+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:20.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:20.535+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:21.305 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:21.303+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:22.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.854+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:22.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:22.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:22.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:22.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:22.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.922+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:22.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.922+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:22.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.922+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:22.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.922+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:22.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:22.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:22.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.954+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:22.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.954+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:22.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.954+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:22.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:22.962+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:23.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.142+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:23.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:23.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:23.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:23.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:23.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:23.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.194+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:23.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:23.222+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:23.496 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:55:23.496 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:55:23.505 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:55:23.530 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:55:24.941 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:55:24.941 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:55:24.946 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244924 2023-10-14T00:55:24.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:55:24.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:55:24.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:55:24.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:55:27.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:27.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:27.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:27.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:27.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:27.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:27.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:27.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:27.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:27.554+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:27.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:27.554+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:27.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:27.554+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:28.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.674+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:55:28.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.674+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:55:28.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.674+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:28.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:28.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:28.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:28.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:28.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:28.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.690+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:28.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.690+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:28.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.690+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:28.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.870+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:28.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:28.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:28.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:28.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:28.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:28.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.910+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:28.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.910+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:28.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:28.910+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:28.942 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:28.942+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:28.976 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:28.974+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:29.366 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:29.366+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:29.376 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:29.370+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:30.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:30.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:30.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:30.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:30.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:30.974+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:32.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:32.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:32.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:32.954+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:33.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:32.998+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:33.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:33.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:33.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:33.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.238+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:33.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:33.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:33.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:33.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:33.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.726+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:33.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:33.726+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:33.915 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:33.914+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:34.955 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:55:34.964 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:55:34.971 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:55:34.977 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:55:35.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:35.910+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:36.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:55:36.693 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:55:36.694 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244936 2023-10-14T00:55:36.695 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:55:36.695 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:55:36.695 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:55:36.695 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:55:37.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.622+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:37.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:37.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:37.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:37.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:37.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:37.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:37.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:37.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:37.678+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:38.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.194+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:38.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:38.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:38.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:38.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:38.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.250+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:38.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.250+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:38.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:38.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:41.857 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:41.854+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:55:41.894 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:41.894+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:42.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:42.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:42.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:42.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:42.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:42.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.390+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:42.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.390+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:42.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:42.394+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:44.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:44.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:44.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:44.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:44.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:44.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:44.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:46.703 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:55:46.704 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:55:46.705 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:55:46.718 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:55:47.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:47.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:47.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:47.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:47.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:47.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:47.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:47.434+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:47.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:47.434+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:47.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:47.446+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:48.018 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:55:48.018 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:55:48.019 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244948 2023-10-14T00:55:48.019 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:55:48.019 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:55:48.019 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:55:48.020 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:55:49.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:49.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:49.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:49.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:49.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:49.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.298+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:49.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.298+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:49.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:49.298+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:52.120 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:52.118+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:54.246 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:54.246+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:54.248 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:54.246+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:55:54.270 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:54.270+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:54.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:54.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:54.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:54.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:54.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:54.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:54.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:54.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:54.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:54.782+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:54.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:54.782+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:54.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:54.794+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:55.241 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:55:55.238+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:55:55.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.294+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:55.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.302+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:55.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:55.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:55.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:55.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.338+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:55.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.338+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:55.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.350+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:55.611 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:55.610+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:55.612 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:55.610+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:55.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.982+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:55.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:55.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:55.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:55.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:55.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.990+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:55.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.990+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:55.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:55.990+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:56.700 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:56.698+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:56.706 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:56.702+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:58.024 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:55:58.025 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:55:58.029 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:55:58.035 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:55:58.209 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:58.206+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:55:58.225 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:58.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:55:58.242 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:58.242+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:58.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:58.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:58.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:58.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:58.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:58.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.534+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:58.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.534+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:58.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:58.534+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:58.871 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:55:58.869+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:55:59.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.173+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:59.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:59.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:59.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:59.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:59.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:59.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.197+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:59.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.197+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:59.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.253+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:59.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.277+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:55:59.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.277+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:55:59.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.277+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:55:59.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:59.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:59.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:59.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:59.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:59.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.329+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:59.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.329+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:59.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.329+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:55:59.603 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:55:59.604 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:55:59.604 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244959 2023-10-14T00:55:59.604 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:55:59.605 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:55:59.605 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:55:59.605 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:55:59.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:55:59.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:55:59.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:55:59.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:55:59.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:55:59.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.933+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:55:59.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.933+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:55:59.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:55:59.957+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:01.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:01.457+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:01.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:01.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:01.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:01.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:01.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:01.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:01.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:01.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:01.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:01.481+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:01.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:01.485+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:05.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.177+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:05.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:05.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:05.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:05.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:05.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:05.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.233+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:05.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.233+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:05.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:05.249+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:06.121 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:06.117+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:56:06.130 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:06.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:06.132 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:06.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:07.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:07.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:07.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:07.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:07.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:07.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:07.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:07.197+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:08.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:08.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:08.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:08.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:08.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:08.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:08.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:08.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:08.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:08.665+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:08.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:08.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:09.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.561+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:09.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:09.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:09.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:09.633 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:56:09.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:09.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:09.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.633+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:09.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.633+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:09.635 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:56:09.636 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:56:09.648 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:56:09.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:09.649+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:09.776 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:09.773+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:56:09.805 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:09.805+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:09.807 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:09.805+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:11.115 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:56:11.116 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:56:11.117 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244971 2023-10-14T00:56:11.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:56:11.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:56:11.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:56:11.122 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:56:11.275 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:11.273+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:12.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:12.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:12.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:12.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:12.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.945+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:12.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.945+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:12.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:13.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:12.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:13.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:13.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:13.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.017+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:13.063 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:13.061+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:13.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:13.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:13.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:13.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:13.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.841+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:13.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.841+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:13.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:13.849+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:14.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.281+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:14.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:14.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:14.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:14.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:14.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:14.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:14.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:14.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:14.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:14.464 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:14.461+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:18.352 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:18.349+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:56:18.367 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:18.365+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:18.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.397+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:56:18.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.401+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.437+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:18.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.437+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:18.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:18.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:19.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.085+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:19.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:19.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:19.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:19.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:19.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.117+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:19.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.117+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:19.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:19.117+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:20.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:20.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:20.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:20.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:20.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:20.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:20.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:20.317+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:20.323 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:20.317+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:20.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:20.325+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:21.139 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:56:21.140 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:56:21.140 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:56:21.142 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:56:22.543 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:56:22.543 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:56:22.543 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244982 2023-10-14T00:56:22.544 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:56:22.544 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:56:22.544 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:56:22.544 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:56:23.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:23.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:23.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:23.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:23.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:23.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:23.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:23.969+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:23.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:23.969+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:23.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:23.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:26.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.369+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:26.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:26.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:26.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:26.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:26.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.481+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:26.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.481+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:26.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.485+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:26.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.597+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:26.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:26.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:26.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:26.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:26.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:26.673+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:26.921 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:26.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:26.929 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:26.925+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:27.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:27.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:27.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:27.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:27.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:27.449+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:27.989 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:27.985+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:28.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:28.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:31.638 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:31.633+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:32.546 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:56:32.550 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:56:32.552 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:56:32.573 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:56:33.648 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:56:33.657 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:56:33.660 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697244993 2023-10-14T00:56:33.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:56:33.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:56:33.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:56:33.661 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:56:33.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:33.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:33.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:33.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:33.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:33.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.905+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:33.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.905+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:33.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:33.913+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:34.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.849+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:34.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:34.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:34.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:34.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:34.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:34.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.877+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:34.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.877+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:34.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:34.917+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:36.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:36.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:36.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:36.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:36.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:36.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.116+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:36.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.116+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:36.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.116+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:36.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:36.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:36.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:36.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:36.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.712+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:36.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.712+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:36.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:36.716+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:37.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:37.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:37.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:37.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:37.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:37.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.208+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:37.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.208+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:37.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.280+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:37.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.324+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:56:37.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.324+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:56:37.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.332+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:37.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:37.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:37.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:37.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:37.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:37.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:37.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:37.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.528+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:37.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.684+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:37.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:37.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:37.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:37.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:37.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:37.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.760+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:37.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.760+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:37.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:37.760+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:39.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:39.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:39.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:39.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:40.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:40.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:40.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:40.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:40.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:40.004+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:40.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:40.004+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:40.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:40.004+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:40.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:40.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:40.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:40.860+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:42.562 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:42.560+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:42.785 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:42.780+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:43.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:43.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:43.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:43.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:43.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.220+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:43.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.220+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:43.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.248+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:43.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:43.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:43.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:43.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:43.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:43.468+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:43.675 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:56:43.677 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:56:43.678 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:56:43.694 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:56:45.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.108+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:45.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:45.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:45.134 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:56:45.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.132+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:45.142 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:56:45.148 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245005 2023-10-14T00:56:45.149 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:56:45.149 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:56:45.149 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:56:45.149 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:56:45.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:45.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.148+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:45.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.148+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:45.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:45.148+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:45.763 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:45.760+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:56:45.783 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:45.780+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:45.802 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:45.796+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:47.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.284+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:47.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:47.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:47.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:47.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:47.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:47.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.304+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:47.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.304+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:47.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.328+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:47.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.616+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:47.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:47.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:47.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:47.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:47.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.664+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:47.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.664+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:47.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:47.664+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:48.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.348+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:48.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:48.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:48.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:48.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:48.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:48.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:48.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:48.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:48.392+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:50.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:50.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:50.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:50.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:50.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:50.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.600+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:50.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.600+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:50.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.604+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:50.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:50.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:50.836+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:51.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:51.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.324+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:51.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.788+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:51.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:51.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:51.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:51.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:51.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:51.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.840+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:51.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.840+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:51.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:51.844+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:52.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.364+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:56:52.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:52.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:52.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:52.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:52.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:52.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:52.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:52.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:52.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:54.369 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:54.364+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:56:54.378 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:54.376+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:55.158 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:56:55.161 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:56:55.166 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:56:55.166 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:56:56.305 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:56:56.306 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:56:56.306 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245016 2023-10-14T00:56:56.307 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:56:56.307 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:56:56.308 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:56:56.308 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:56:58.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:58.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:56:58.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:58.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:56:58.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:56:58.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.284+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:56:58.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.284+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:56:58.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.284+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:58.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:58.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:56:59.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:59.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:56:59.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:59.008+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:56:59.570 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:59.564+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:56:59.580 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:56:59.568+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:56:59.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:56:59.640+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:00.533 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:00.528+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:57:00.542 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:00.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:00.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:00.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:00.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:00.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:00.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:00.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:00.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:00.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:00.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:00.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:00.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:00.988+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:03.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:03.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:03.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:03.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:03.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:03.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:03.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.476+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:03.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:03.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:03.480 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:03.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:03.506 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:03.508+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:04.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:04.020+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:04.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:04.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:04.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:04.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:04.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:04.044+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:04.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:04.044+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:04.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:04.052+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:06.313 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:57:06.330 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:57:06.331 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:57:06.332 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:57:06.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.496+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:06.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:06.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:06.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:06.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:06.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.536+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:06.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.536+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:06.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:06.536+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:07.685 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:57:07.686 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:57:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245027 2023-10-14T00:57:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:57:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:57:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:57:07.691 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:57:10.291 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:10.291+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:10.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.355+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:10.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:10.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:10.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:10.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:10.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:10.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.387+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:10.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.387+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:10.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.387+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:10.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:10.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:10.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:10.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:10.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:10.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.919+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:10.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:10.931+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:11.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.583+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:11.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:11.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:11.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:11.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:11.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.643+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:11.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.643+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:11.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:11.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:14.756 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:14.751+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:57:14.795 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:14.795+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:16.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:16.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:17.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:17.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:17.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:17.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:17.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:17.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:17.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:17.071+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:17.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:17.071+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:17.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:17.075+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:17.695 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:57:17.695 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:57:17.698 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:57:17.716 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:57:19.110 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:57:19.111 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:57:19.112 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245039 2023-10-14T00:57:19.112 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:57:19.112 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:57:19.112 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:57:19.112 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:57:19.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:19.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:19.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:19.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:19.212 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:19.211+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:19.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:19.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.207+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:19.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.207+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:19.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.215+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:19.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:19.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:19.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:19.711+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:20.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.403+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:20.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:20.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:20.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:20.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:20.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:20.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.447+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:20.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.447+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:20.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:20.447+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:21.923 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:21.923+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:57:21.923 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:21.923+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:22.158 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:22.155+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:22.159 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:22.159+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:23.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:23.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:23.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:23.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:23.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:23.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:23.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:23.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:23.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:23.591+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:23.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:23.591+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:23.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:23.591+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:24.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.331+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:24.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:24.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:24.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:24.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:24.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:24.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.591+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:24.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.591+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:24.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:24.603+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:25.678 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:25.667+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:25.694 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:25.691+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:25.766 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:25.767+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:26.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:26.359+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:27.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.191+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:27.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:27.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:27.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:27.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:27.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:27.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.619+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:27.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.619+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:27.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:27.623+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:28.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:28.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:28.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:28.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:28.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:28.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:28.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:28.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:28.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:28.707+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:28.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:28.707+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:28.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:28.715+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:29.140 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:57:29.145 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:57:29.158 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:57:29.169 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:57:30.007 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:30.007+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:30.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.347+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:30.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:30.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:30.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:30.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:30.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:30.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.391+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:30.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:30.628 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:57:30.629 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:57:30.629 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245050 2023-10-14T00:57:30.630 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:57:30.630 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:57:30.630 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:57:30.631 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:57:30.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.883+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:30.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:30.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:30.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:30.927+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:31.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:31.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:31.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:31.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:31.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:31.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.095+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:31.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.107+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:31.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:31.127+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:32.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:32.351+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:32.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:32.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:32.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:32.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:32.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:32.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:32.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:32.431+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:32.439 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:32.439+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:57:32.457 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:32.455+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:57:32.457 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:32.455+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:32.460 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:32.459+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:32.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:32.607+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:32.919 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:32.919+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:32.959 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:32.959+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:33.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:33.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:33.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:33.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:33.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.487+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:33.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.487+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:33.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.499+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:33.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.535+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:33.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:33.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:33.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:33.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:33.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:33.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.579+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:33.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.579+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:33.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:33.579+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:34.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:34.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:34.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:34.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.263+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:34.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:34.568 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:34.567+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:57:34.598 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:34.595+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:34.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:34.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:34.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:34.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:34.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.623+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:34.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.623+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:34.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:34.627+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:36.412 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:36.411+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:36.439 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:36.439+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:36.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.543+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:57:36.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.543+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:57:36.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.563+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:36.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:36.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:36.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:36.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:36.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:36.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.595+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:36.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.599+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:36.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:36.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:37.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:37.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:37.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:37.715+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:39.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:39.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:39.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:39.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:39.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:39.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:39.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:39.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:39.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:39.235+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:39.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:39.235+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:39.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:39.235+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:40.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:40.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:40.644 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:57:40.644 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:57:40.645 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:57:40.669 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:57:41.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.743+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:57:41.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.743+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:57:41.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.787+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:41.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:41.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:41.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:41.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:41.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:41.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:41.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:41.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:42.099 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:57:42.099 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:57:42.099 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245062 2023-10-14T00:57:42.100 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:57:42.100 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:57:42.100 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:57:42.100 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:57:42.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:42.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:42.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:42.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:42.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:42.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:42.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:42.623+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:43.549 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:57:43.547+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:57:44.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:44.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:44.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:44.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:44.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:44.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.171+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:44.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.199+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:44.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:44.199+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:45.222 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:45.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:45.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.250+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:45.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:45.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:45.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:45.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:45.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:45.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.310+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:45.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.310+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:45.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:45.314+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:46.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:46.106+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:46.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:46.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:46.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:46.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:46.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:46.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:46.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:46.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:46.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:46.178+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:46.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:46.178+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:47.163 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:47.158+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:47.231 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:47.230+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:48.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.102+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:48.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:48.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:48.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:48.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:48.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:48.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.178+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:48.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.178+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:48.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:48.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:48.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:48.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:48.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.594+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:48.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.594+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:48.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:48.618+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:49.496 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:49.494+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T00:57:49.524 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:49.522+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:49.552 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:49.550+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:50.086 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:50.082+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:50.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.914+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:50.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:50.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:50.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:50.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:50.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:50.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.974+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:50.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.978+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:50.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:50.978+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:51.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:51.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:51.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:51.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:51.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:51.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:51.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:51.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:51.162+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:52.101 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:57:52.101 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:57:52.106 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:57:52.123 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:57:53.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:53.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:53.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.014+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:53.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.014+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:53.403 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:57:53.407 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:57:53.409 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245073 2023-10-14T00:57:53.409 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:57:53.409 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:57:53.409 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:57:53.410 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:57:53.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:53.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:53.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:53.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:53.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.722+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:53.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.722+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:53.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:53.722+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:54.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.842+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:54.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:54.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:54.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:54.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:54.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:54.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.882+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:54.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.882+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:54.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:54.898+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:54.994 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:54.990+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:57:55.085 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:55.078+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:57:55.098 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:57:55.098+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:57:55.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:55.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:55.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:55.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:55.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.466+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:55.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.466+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:55.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.474+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:55.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:55.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:55.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:55.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:55.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:55.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:55.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.894+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:55.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:55.894+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:56.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:56.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:56.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:56.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:56.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:56.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:56.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:56.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:56.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:56.294+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:57.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:57.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:57.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:57.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:57.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:57.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:57.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:57.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:57.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:57.638+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:57.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:57.638+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:57.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:57.646+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:58.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:58.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:58.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:58.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:58.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.230+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:58.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:58.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:58.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:58.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:58.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.410+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:58.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.410+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:58.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.414+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:58.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:58.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:58.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:58.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:58.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.506+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:58.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.506+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:58.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.506+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:58.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.710+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:58.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:58.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:58.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:58.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:58.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.750+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:58.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.750+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:57:58.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:58.758+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:59.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:59.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:59.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:59.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:59.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:59.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.366+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:57:59.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.738+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:57:59.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:57:59.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:57:59.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:57:59.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:57:59.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:57:59.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.822+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:57:59.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:57:59.826+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:00.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.154+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:00.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:00.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:00.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:00.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:00.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:00.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.174+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:00.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.174+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:00.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.174+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:00.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:00.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:01.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:01.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:01.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:01.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.038+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:01.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.038+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:01.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:01.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:01.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:01.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:01.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:01.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:01.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:01.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:01.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:01.314+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:02.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:02.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:02.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:02.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:02.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:02.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:02.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:02.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:02.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:02.478+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:02.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:02.478+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:02.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:02.490+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:03.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:03.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:03.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:03.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:03.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.114+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:03.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.114+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:03.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:03.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:03.418 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:58:03.418 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:58:03.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:03.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:03.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.418+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:03.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.418+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:03.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:03.422+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:03.446 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:58:03.449 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:58:03.619 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:03.614+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:58:03.666 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:03.666+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:04.527 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:58:04.526+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:58:04.944 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:58:04.945 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:58:04.945 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245084 2023-10-14T00:58:04.945 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:58:04.946 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:58:04.946 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:58:04.946 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:58:05.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:05.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:05.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:05.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:05.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:05.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:05.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:05.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:05.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:05.838+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:05.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:05.838+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:05.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:05.862+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:06.147 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:06.146+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:58:06.155 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:06.154+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:58:06.155 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:06.154+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:06.619 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:06.618+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:58:06.645 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:06.642+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:07.530 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:07.526+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:08.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:08.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:08.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:08.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:08.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:08.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:08.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:08.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:08.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:08.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:08.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:08.902+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:09.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.198+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:09.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:09.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:09.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:09.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:09.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:09.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.266+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:09.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.266+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:09.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.290+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:09.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.670+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:09.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.670+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:09.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:09.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:09.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:09.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:09.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:09.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:09.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.702+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:09.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:09.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:09.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:09.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:09.886+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:12.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.290+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:12.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:12.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:12.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:12.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:12.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.362+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:12.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.362+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:12.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.370+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:12.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.370+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:12.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:12.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:12.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:12.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:12.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:12.442+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:14.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:14.094+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:14.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:14.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:14.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:14.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:14.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:14.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:14.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:14.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:14.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:14.174+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:14.951 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:58:14.952 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:58:14.956 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:58:14.956 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:58:15.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.126+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:15.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:15.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:15.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:15.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:15.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:15.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.214+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:15.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.214+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:15.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:15.222+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:16.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:16.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:16.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:16.078+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:16.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:16.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:16.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:16.110+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:16.110+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:16.110+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:16.397 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:58:16.398 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:58:16.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245096 2023-10-14T00:58:16.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:58:16.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:58:16.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:58:16.407 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:58:17.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.402+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:58:17.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.402+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:58:17.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.406+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:17.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:17.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:17.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:17.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:17.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:17.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.438+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:17.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.438+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:17.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:17.446+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:19.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:19.386+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:20.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:20.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:20.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:20.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:20.537+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:20.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:20.545+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:21.676 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:21.673+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:58:21.718 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:21.709+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:23.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:23.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:23.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:23.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:23.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:23.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:23.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:23.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:23.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:23.329+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:23.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:23.329+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:23.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:23.365+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:23.796 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:58:23.793+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:58:24.602 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:24.597+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:24.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:24.881+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:24.941 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:24.937+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:58:26.430 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:58:26.430 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:58:26.434 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:58:26.436 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:58:26.575 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:26.573+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:58:28.007 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:58:28.007 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:58:28.008 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245108 2023-10-14T00:58:28.008 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:58:28.008 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:58:28.008 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:58:28.009 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:58:28.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:28.469+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:28.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:28.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:28.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:28.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:28.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:28.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:28.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:28.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:28.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:28.525+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:28.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:28.525+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:29.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.021+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:29.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:29.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:29.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:29.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:29.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:29.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.061+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:29.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.061+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:29.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:29.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:29.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.285+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:29.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.497+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:29.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:29.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:29.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:29.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:29.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:29.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.605+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:29.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.605+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:29.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:29.605+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:32.149 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:32.145+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:58:32.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:32.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:32.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.365+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:32.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:32.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:32.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:32.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.385+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:32.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.449+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:32.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:32.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:32.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:32.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:32.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.525+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:32.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:32.525+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:33.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:33.069+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:58:33.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:33.081+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:58:33.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:33.109+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:33.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:33.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:33.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:33.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:33.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:33.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:33.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:33.165+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:34.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:34.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:34.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:34.081+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:35.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:35.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:35.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:35.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:35.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:35.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:35.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:35.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:35.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:35.781+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:35.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:35.781+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:35.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:35.785+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:37.863 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:37.857+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:58:38.028 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:58:38.036 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:58:38.036 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:58:38.066 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:58:39.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.305+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:39.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:39.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:39.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:39.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:39.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:39.414 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:58:39.414 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:58:39.415 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245119 2023-10-14T00:58:39.415 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:58:39.416 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:58:39.416 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:58:39.416 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:58:39.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:39.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:39.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:39.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:39.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:39.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.657+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:39.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.665+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:39.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:39.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:40.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:40.393+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:40.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:40.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:40.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:40.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:40.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:40.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:40.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:40.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:41.582 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:41.573+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:58:41.582 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:41.577+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:42.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:42.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:42.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:42.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:42.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:42.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.913+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:42.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.913+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:42.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:42.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:43.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:43.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:43.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:43.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:43.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:43.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.369+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:43.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:43.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:43.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:43.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:43.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:43.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:43.673+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:44.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:44.273+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:44.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:44.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:44.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:44.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:44.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:44.305+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:45.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:45.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:45.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:45.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:45.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:45.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:45.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:45.389+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:46.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:46.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:46.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:46.169+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:47.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:47.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:47.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:47.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:47.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:47.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:47.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:47.461+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:49.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:48.997+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:49.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:49.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:49.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:49.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:49.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.093+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:49.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.093+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:49.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:49.435 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:58:49.436 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:58:49.438 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:58:49.473 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:58:49.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:49.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:49.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:49.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:49.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.521+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:49.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.521+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:49.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:49.529+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:50.776 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:58:50.776 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:58:50.777 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245130 2023-10-14T00:58:50.777 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:58:50.777 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:58:50.778 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:58:50.778 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:58:53.200 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:53.197+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:58:53.486 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:58:53.481+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:58:54.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.453+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:58:54.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:54.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:58:54.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:54.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:54.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:54.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.521+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:54.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.521+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:54.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.561+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:58:54.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:54.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:54.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:56.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:56.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:58:56.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:56.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:58:56.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:56.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:58:56.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:56.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:58:56.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:56.716+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:58:56.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:56.716+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:58:56.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:58:56.720+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:00.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:00.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:00.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:00.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:00.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:00.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.036+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:00.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.036+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:00.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:00.040+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:00.781 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:59:00.782 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:59:00.782 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:59:00.789 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:59:01.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.096+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:01.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:01.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:01.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:01.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:01.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:01.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.168+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:01.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.168+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:01.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.188+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:01.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:01.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.644+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:01.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:01.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:01.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:01.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:01.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.700+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:01.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.700+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:01.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:01.700+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:02.231 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:59:02.233 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:59:02.234 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245142 2023-10-14T00:59:02.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:59:02.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:59:02.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:59:02.235 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:59:04.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.276+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T00:59:04.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.276+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T00:59:04.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.276+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:04.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:04.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:04.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:04.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:04.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.300+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:04.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.300+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:04.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.300+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:04.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:04.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.832+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:04.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:04.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:04.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:04.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:04.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.884+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:04.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.884+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:04.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:04.896+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:05.855 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:59:05.852+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:59:06.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:06.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:06.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:06.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:06.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:06.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:06.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:06.272+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:08.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:08.436+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:09.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:09.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:10.453 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:10.448+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:10.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:10.816+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:10.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:10.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:10.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:10.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:10.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:10.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:10.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:10.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:10.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:10.880+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:12.246 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:59:12.246 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:59:12.246 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:59:12.269 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:59:13.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:13.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:13.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:13.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:13.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:13.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:13.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:13.208+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:13.545 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:59:13.545 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:59:13.546 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245153 2023-10-14T00:59:13.547 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:59:13.547 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:59:13.547 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:59:13.547 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:59:14.488 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:14.484+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:59:14.522 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:14.516+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:14.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.812+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:14.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:14.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:14.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:14.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:14.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:14.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.876+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:14.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.876+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:14.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:14.876+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:15.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:15.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:15.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:15.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:15.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:15.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.472+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:15.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.472+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:15.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:15.480+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:16.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.068+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:16.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:16.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.096+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:16.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.232+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:16.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:16.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:16.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:16.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:16.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:16.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.288+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:16.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.288+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:16.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.288+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:16.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:16.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:16.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:16.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:16.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.872+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:16.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.880+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:16.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:16.916+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:17.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.428+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:17.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:17.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:17.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:17.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:17.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:17.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:17.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.476+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:17.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:17.480+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:19.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:19.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:19.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:19.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:19.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:19.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:19.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:19.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:19.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:19.668+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:19.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:19.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:20.209 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:20.204+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:59:20.209 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:20.204+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:21.434 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:21.428+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:59:21.842 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:21.836+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:22.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:22.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:22.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:22.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:22.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.144+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:22.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.144+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:22.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.144+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:22.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:22.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:22.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:22.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:22.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:22.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.352+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:22.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.352+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:22.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:22.364+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:23.137 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:23.136+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:59:23.148 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:23.148+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:23.560 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:59:23.563 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:59:23.565 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:59:23.569 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:59:24.777 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:59:24.777 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:59:24.778 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245164 2023-10-14T00:59:24.778 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:59:24.778 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:59:24.779 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:59:24.779 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:59:24.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:24.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:24.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:24.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:24.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:24.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:24.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:24.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:24.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:24.944+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:24.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:24.944+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:24.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:24.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:25.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:25.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:25.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:25.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:25.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:25.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:25.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:25.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:25.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:25.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:25.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:25.772+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:25.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:25.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:26.303 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:26.296+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:59:26.307 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:26.308+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:27.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.020+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:27.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:27.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:27.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:27.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:27.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.104+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:27.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.104+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:27.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:27.124+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:28.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:28.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:28.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:28.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:28.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:28.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.952+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:28.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.952+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:28.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:28.960+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:29.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:29.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:29.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:29.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:29.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:29.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:29.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:29.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:29.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:29.608+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:29.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:29.608+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:29.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:29.640+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:30.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:30.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:30.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:30.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:30.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:30.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:30.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:30.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:30.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:30.124+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:30.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:30.124+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:30.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:30.132+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:32.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:32.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:32.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:32.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:32.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:32.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:32.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:32.015+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:32.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:32.023+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:32.429 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:32.427+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:59:32.444 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:32.443+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:33.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:33.871+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:34.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:34.103+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:34.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:34.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:34.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:34.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:34.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:34.143+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:34.282 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:34.279+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:59:34.781 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:59:34.781 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:59:34.781 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:59:34.814 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:59:35.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:35.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:35.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:35.455+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:35.879 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:59:35.879 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:59:35.880 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245175 2023-10-14T00:59:35.880 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:59:35.880 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:59:35.880 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:59:35.880 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:59:35.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:35.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:35.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:35.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:35.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:35.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:35.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:35.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:35.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:35.947+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:37.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.247+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:37.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:37.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:37.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:37.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:37.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:37.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.287+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:37.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:37.307+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:37.867 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:37.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T00:59:37.868 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:37.867+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T00:59:37.904 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T00:59:37.903+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T00:59:38.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:38.791+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:38.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:38.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:38.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:38.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:38.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:38.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:38.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:38.819+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:44.826 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:59:44.823+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T00:59:44.832 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T00:59:44.831+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T00:59:45.882 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:59:45.889 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:59:45.890 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:59:45.899 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:59:46.945 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:59:46.946 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:59:46.946 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245186 2023-10-14T00:59:46.946 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:59:46.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:59:46.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:59:46.947 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T00:59:49.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:49.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:49.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:49.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:49.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:49.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:49.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:49.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:49.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:49.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:49.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:49.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:49.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:49.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:50.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:50.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:50.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:50.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:50.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:50.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:50.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:50.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:50.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:50.879+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:50.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:50.879+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:50.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:50.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:52.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.151+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:52.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:52.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:52.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:52.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:52.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:52.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.191+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:52.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.211+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:52.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.211+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:52.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.831+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:52.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:52.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:52.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:52.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:52.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:52.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.891+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:52.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.891+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:52.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:52.907+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:53.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:53.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:53.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:53.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:53.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:53.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.895+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:53.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.895+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:53.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:53.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:55.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:55.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:55.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:55.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:55.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:55.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:55.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:55.583+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:56.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.067+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T00:59:56.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:56.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:56.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:56.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:56.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:56.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.159+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:56.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.159+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:56.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:56.167+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T00:59:56.966 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T00:59:56.976 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T00:59:56.989 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T00:59:56.991 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T00:59:57.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:57.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T00:59:57.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:57.231+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T00:59:57.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:57.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T00:59:57.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:57.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T00:59:57.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:57.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T00:59:57.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:57.239+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T00:59:57.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T00:59:57.239+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T00:59:58.482 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T00:59:58.483 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T00:59:58.483 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245198 2023-10-14T00:59:58.483 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T00:59:58.483 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T00:59:58.484 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T00:59:58.484 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:00:01.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:01.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:01.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:01.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:01.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:01.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.239+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:01.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.239+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:01.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.239+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:01.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:01.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:01.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:01.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:01.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:01.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:01.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:01.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:01.503+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:02.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:02.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:02.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:02.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:02.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:02.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:02.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:02.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:02.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:02.191+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:02.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:02.199+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:02.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:02.203+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:05.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:05.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:05.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:05.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:05.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:05.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:05.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:05.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:05.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:05.483+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:05.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:05.483+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:05.654 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:05.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:00:05.655 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:05.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:06.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:06.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:06.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:06.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:06.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:06.583+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:06.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:06.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:06.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:06.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:08.486 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T01:00:08.488 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T01:00:08.494 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T01:00:08.494 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T01:00:09.877 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T01:00:09.878 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T01:00:09.878 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245209 2023-10-14T01:00:09.878 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T01:00:09.879 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T01:00:09.879 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T01:00:09.879 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:00:10.428 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:10.426+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:10.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.778+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:10.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:10.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:10.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:10.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:10.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:10.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.842+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:10.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.842+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:10.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:10.854+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:12.637 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:12.634+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:13.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:13.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:13.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:13.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:13.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:13.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:13.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:13.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:13.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:13.442+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:13.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:13.442+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:13.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:13.454+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:14.342 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:14.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:15.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.238+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:00:15.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.238+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:00:15.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.238+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:15.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:15.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:15.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:15.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:15.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:15.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.262+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:15.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.262+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:15.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:15.262+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:16.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:16.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:16.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.066+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:16.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:16.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:16.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:16.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:16.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:16.106+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:17.469 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:17.466+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:00:17.478 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:17.474+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:00:17.481 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:17.478+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:17.495 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:17.494+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:17.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.730+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:17.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.730+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:17.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:17.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:17.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:17.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.754+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:17.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.754+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:17.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:17.754+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:19.886 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T01:00:19.888 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T01:00:19.901 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T01:00:19.908 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T01:00:19.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:19.946+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:19.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:19.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:19.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:19.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:19.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:19.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:20.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:20.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:20.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.014+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:20.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.014+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:20.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:20.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:20.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:20.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:20.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:20.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.226+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:20.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:20.226+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:21.276 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T01:00:21.276 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T01:00:21.277 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245221 2023-10-14T01:00:21.277 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T01:00:21.277 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T01:00:21.277 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T01:00:21.277 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:00:25.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:25.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:25.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:25.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:26.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:26.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:26.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:26.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:26.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:26.014+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:26.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:26.014+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:26.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:26.034+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:26.734 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:26.730+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:00:26.734 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:26.730+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:27.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.342+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:27.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:27.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:27.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:27.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:27.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:27.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.358+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:27.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.358+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:27.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:27.370+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:28.424 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:28.422+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:28.434 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:28.426+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:29.392 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:29.390+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:29.751 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:29.750+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:30.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:30.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:30.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:30.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:30.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.234+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:30.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.234+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:30.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.234+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:30.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:30.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:30.966+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:31.287 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T01:00:31.290 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T01:00:31.291 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T01:00:31.298 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T01:00:32.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:32.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:32.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:32.542+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:32.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:32.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:32.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:32.566+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:32.645 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T01:00:32.662 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T01:00:32.663 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245232 2023-10-14T01:00:32.663 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T01:00:32.663 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T01:00:32.663 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T01:00:32.664 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:00:33.425 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:33.422+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:33.452 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:33.450+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:33.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:33.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:33.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:33.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:33.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:33.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.598+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:33.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.598+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:33.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.614+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:33.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.854+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:33.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:33.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:33.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:33.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:33.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.898+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:33.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.898+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:33.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:33.906+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:36.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.306+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:00:36.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.318+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:36.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:36.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:36.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:36.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:36.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:36.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.422+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:36.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.422+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:36.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:36.434+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:38.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:38.906+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:38.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:38.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:38.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:38.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:38.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:38.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:38.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:38.926+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:40.362 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:40.358+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:00:40.377 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:40.370+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:40.383 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:40.382+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:40.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:40.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:40.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:40.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:40.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:40.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.894+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:40.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.894+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:40.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:40.906+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:41.623 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:41.622+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:00:41.640 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:41.638+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:41.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.790+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:41.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:41.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:41.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:41.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:41.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:41.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.822+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:41.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.822+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:41.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:41.822+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:42.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:42.030+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:42.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:42.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:42.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:42.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:42.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:42.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:42.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:42.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:42.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:42.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:42.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:42.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:42.685 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T01:00:42.685 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T01:00:42.698 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T01:00:42.705 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T01:00:43.808 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T01:00:43.808 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T01:00:43.809 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245243 2023-10-14T01:00:43.809 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T01:00:43.809 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T01:00:43.809 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T01:00:43.809 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:00:44.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:44.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:44.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:44.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:44.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.626+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:44.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.626+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:44.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.626+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:44.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.874+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:44.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:44.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:44.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:44.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:44.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.918+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:44.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.946+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:44.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:44.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:44.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:44.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:44.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:44.986+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:46.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.118+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:46.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:46.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:46.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:46.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.154+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:46.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:46.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:46.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:46.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:46.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:46.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.594+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:46.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.594+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:46.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.606+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:46.700 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:46.698+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:00:46.721 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:46.718+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:46.746 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:46.742+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:46.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:46.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:46.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:46.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:46.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:46.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.978+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:46.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.978+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:47.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:46.998+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:48.369 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:48.365+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:00:48.412 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:48.409+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:51.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.569+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:51.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.585+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:51.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:51.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:51.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:51.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.641+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:51.810 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:51.805+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:00:51.820 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:51.813+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:51.826 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:51.821+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:00:51.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.857+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:51.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:51.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:51.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:51.885+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:53.811 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T01:00:53.812 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T01:00:53.821 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T01:00:53.834 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T01:00:54.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.085+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:54.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:54.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:54.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:54.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:54.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:54.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:54.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:54.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:54.185+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:55.152 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T01:00:55.162 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T01:00:55.163 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245255 2023-10-14T01:00:55.164 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T01:00:55.164 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T01:00:55.164 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T01:00:55.164 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:00:55.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:55.309+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:55.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:55.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:55.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:55.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:55.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:55.361+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:56.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.297+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:56.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:56.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:00:56.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:56.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:56.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:56.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:56.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.361+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:56.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:56.365+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:58.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.073+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:58.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:58.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:58.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:58.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:58.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.105+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:58.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.105+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:58.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:58.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:58.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:58.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.333+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:58.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.333+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:58.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.669+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:58.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:58.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:58.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:58.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:58.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.689+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:58.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:58.689+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:59.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.189+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:00:59.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:00:59.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:00:59.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:00:59.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:00:59.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.249+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:00:59.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.249+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:00:59.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:00:59.249+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:00:59.964 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:59.961+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:00:59.982 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:59.981+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:00:59.983 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:00:59.981+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:01.043 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:01.041+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:01:03.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:03.753+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:04.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:04.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:04.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:04.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:04.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.293+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:04.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.293+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:04.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.313+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:04.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.593+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:04.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:04.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:04.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:04.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:04.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:04.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.649+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:04.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.649+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:04.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:04.649+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:05.166 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T01:01:05.183 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T01:01:05.195 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T01:01:05.215 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T01:01:05.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:05.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:05.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:05.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:05.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:05.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.741+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:05.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.753+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:05.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:05.777+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:06.512 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T01:01:06.512 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T01:01:06.515 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245266 2023-10-14T01:01:06.515 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T01:01:06.516 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ -n 172.21.15.183:0/4061210612 ]] 2023-10-14T01:01:06.516 INFO:tasks.workunit.client.0.smithi183.stderr:+ [[ 172.21.15.183:0/4061210612 != \1\7\2\.\2\1\.\1\5\.\1\8\3\:\0\/\4\0\6\1\2\1\0\6\1\2 ]] 2023-10-14T01:01:06.516 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:01:08.085 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:08.081+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:08.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:08.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:08.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:08.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:08.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:08.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.409+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:08.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.409+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:08.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:08.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:12.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:12.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:12.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:12.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:12.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:12.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.465+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:12.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.465+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:12.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:12.559 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:12.557+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:12.559 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:12.557+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:12.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.649+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:01:12.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.649+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:01:12.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.653+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:12.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:12.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:12.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:12.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:12.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:12.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:12.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:12.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:12.705+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:15.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.033+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:15.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:15.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:15.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:15.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:15.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.089+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:15.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.089+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:15.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.089+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:15.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:15.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:15.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.889+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:15.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.889+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:15.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:15.897+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:16.252 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:16.249+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:01:16.255 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:16.253+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:16.288 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:16.281+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:16.529 INFO:tasks.workunit.client.0.smithi183.stderr:++ ceph mgr dump 2023-10-14T01:01:16.530 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq 'select(.name == "rbd_support")' 2023-10-14T01:01:16.531 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq -r '[.addrvec[0].addr, "/", .addrvec[0].nonce|tostring] | add' 2023-10-14T01:01:16.558 INFO:tasks.workunit.client.0.smithi183.stderr:++ jq '.active_clients[]' 2023-10-14T01:01:16.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.873+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:16.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:16.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:16.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:16.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:16.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.925+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:16.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.925+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:16.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:16.925+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:17.059 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:17.057+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:17.072 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:17.069+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:17.959 INFO:tasks.workunit.client.0.smithi183.stderr:+ CLIENT_ADDR=172.21.15.183:0/4061210612 2023-10-14T01:01:17.960 INFO:tasks.workunit.client.0.smithi183.stderr:++ date +%s 2023-10-14T01:01:17.960 INFO:tasks.workunit.client.0.smithi183.stderr:+ CURRENT_TIME=1697245277 2023-10-14T01:01:17.960 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( CURRENT_TIME <= END_TIME )) 2023-10-14T01:01:17.960 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i = 1 )) 2023-10-14T01:01:17.961 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:01:17.961 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:01:18.158 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:01:18.153+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:01:18.159 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:01:18.175 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:01:20.942 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:20.937+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:20.956 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:20.953+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:21.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:21.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:21.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:21.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:21.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:21.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:21.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:21.829+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:21.965 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:21.961+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:01:21.985 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:21.981+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:01:21.985 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:21.981+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:21.986 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:21.981+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:26.134 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:26.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:26.281 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:26.277+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:27.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:27.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:27.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:27.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:27.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:27.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:27.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.416+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:27.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:27.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:27.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:27.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:27.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.916+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:27.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.916+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:27.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.928+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:27.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.928+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:27.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:27.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:27.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:27.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:27.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.944+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:27.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.944+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:27.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:27.944+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:28.179 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:01:28.180 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:01:28.180 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:01:28.303 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:01:28.300+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:01:28.303 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:01:28.328 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:01:28.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:28.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:28.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:28.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:28.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.432+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:28.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:28.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:30.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:30.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:30.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:30.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:30.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:30.452+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:31.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.328+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:31.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:31.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:31.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:31.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.368+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:31.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.424+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:01:31.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:31.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:31.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:31.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:31.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:31.456+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:32.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:32.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:32.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:32.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.412+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:32.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.656+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:32.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:32.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:32.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:32.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:32.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:32.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.772+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:32.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.772+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:32.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:32.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:32.979 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:32.972+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:33.001 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:32.992+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:33.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:33.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:33.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:33.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:33.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:33.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:33.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:33.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:33.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:33.368+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:33.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:33.368+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:33.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:33.372+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:34.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:34.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:34.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:34.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:34.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.748+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:34.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.748+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:34.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.748+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:34.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:34.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:34.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:34.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:34.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.884+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:34.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.884+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:34.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:34.884+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:36.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.208+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:36.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:36.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:36.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:36.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:36.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:36.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.276+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:36.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.276+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:36.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:36.276+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:37.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:37.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:37.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:37.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:37.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:37.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.608+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:37.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.608+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:37.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:37.608+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:38.330 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:01:38.330 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:01:38.330 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:01:38.505 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:01:38.500+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:01:38.505 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:01:38.526 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:01:38.707 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:38.704+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:39.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:39.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:39.247 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:39.244+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:01:39.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:39.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:39.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:39.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.260+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:39.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.260+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:39.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:39.268+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:39.278 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:39.272+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:40.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.116+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:40.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:40.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:40.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:40.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:40.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:40.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.152+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:40.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.152+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:40.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:40.152+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:42.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:42.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:42.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:42.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:42.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:42.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:42.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.552+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:42.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.552+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:42.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:42.552+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:43.999 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:43.996+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:46.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:46.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:46.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:46.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:46.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:46.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:46.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:46.140+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:46.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:46.140+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:47.080 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:47.076+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:01:47.088 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:47.084+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:47.097 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:47.092+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:48.527 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:01:48.527 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:01:48.528 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:01:48.720 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:01:48.722 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:01:48.716+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:01:48.729 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:01:48.793 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:48.784+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:01:48.794 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:48.788+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:48.799 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:48.792+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:01:50.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:50.572+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:50.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:50.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:50.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:50.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:50.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:50.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:50.584+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:50.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:50.584+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:51.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:51.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:51.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:51.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:51.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:51.868+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:01:51.907 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:51.904+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:53.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.516+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:53.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:53.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:53.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:53.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:53.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.544+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:53.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.544+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:53.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:53.552+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:55.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:54.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:55.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:55.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:55.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:55.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.084+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:55.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.084+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:55.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.084+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:55.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.084+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:55.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:55.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:55.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.124+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:55.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.364+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:55.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:55.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:55.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:55.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:55.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:55.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:55.420+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:56.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:56.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:56.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:56.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:56.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:56.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:56.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:56.992+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:57.486 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:01:57.484+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:01:58.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:58.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:58.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:58.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.308+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:58.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.360+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:01:58.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:01:58.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:58.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:58.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:58.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:58.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:58.436+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:01:58.755 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:01:58.755 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:01:58.755 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:01:58.853 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:01:58.852+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:01:58.854 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:01:58.871 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:01:59.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:59.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:01:59.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:59.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:01:59.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:01:59.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:01:59.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:01:59.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:01:59.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:01:59.252+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:00.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:00.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:00.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:00.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:00.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:00.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:00.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:00.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:00.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:00.608+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:00.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:00.608+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:00.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:00.608+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:00.796 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:00.796+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:02:00.832 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:00.832+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:02.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.052+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:02.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:02.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:02.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:02.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:02.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:02.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.136+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:02.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.152+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:02.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:02.172+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:02.533 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:02.532+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:02:02.546 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:02:02.548+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:02:02.550 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:02.548+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:05.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.416+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:05.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:05.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:05.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:05.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:05.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.460+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:05.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.460+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:05.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:05.468+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:08.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.059+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:02:08.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.059+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:02:08.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.087+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:08.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:08.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:08.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:08.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:08.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:08.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.123+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:08.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.123+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:08.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.123+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:08.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:08.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:08.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.671+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:08.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.671+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:08.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.683+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:08.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:08.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:08.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:08.707+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:08.881 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:02:08.882 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:02:08.882 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:02:09.053 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:02:09.051+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:02:09.056 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:02:09.063 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:02:09.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:09.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:09.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:09.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:09.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:09.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.903+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:09.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.903+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:09.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:09.943+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:11.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:11.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:11.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:11.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:11.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:11.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:11.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:11.203+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:12.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:12.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:12.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:12.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:12.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:12.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.211+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:12.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.211+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:12.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.211+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:12.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.947+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:02:12.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.947+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:02:12.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:12.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:12.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:12.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:12.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:13.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:13.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:13.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:13.007+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:13.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:13.007+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:13.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:13.011+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:14.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:14.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:14.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:14.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.167+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:14.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.167+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:14.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.167+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:14.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:14.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:14.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:14.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.571+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:14.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.571+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:14.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.627+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:14.777 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:14.775+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:14.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.787+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:14.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:14.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:14.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:14.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:14.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.819+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:14.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.819+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:14.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.819+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:14.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.907+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:14.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:14.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:14.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:14.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:14.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:14.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.967+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:14.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.967+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:14.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:14.967+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:17.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:17.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:17.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:17.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:17.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:17.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:17.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:17.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:17.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:17.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:17.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:17.379+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:17.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:17.379+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:17.548 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:17.547+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:02:17.550 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:17.547+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:18.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.299+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:02:18.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.299+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:02:18.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.311+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:18.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.311+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:18.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:18.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:18.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:18.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:18.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.411+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:18.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.411+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:18.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:18.419+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:19.065 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:02:19.065 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:02:19.065 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:02:19.241 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:02:19.239+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:02:19.241 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:02:19.246 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:02:20.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:20.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:20.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:20.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:20.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:20.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:20.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:20.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:20.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:20.667+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:21.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:21.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:21.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:21.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:21.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:21.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:21.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:21.955+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:21.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:21.955+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:21.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:21.967+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:22.457 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:22.455+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:22.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:22.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:22.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:22.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:22.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:22.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:22.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:22.679+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:22.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:22.679+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:24.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:24.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:24.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:24.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:24.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:24.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:24.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:24.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:25.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:25.659+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:25.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:25.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:25.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:25.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:25.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:25.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:25.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:25.687+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:25.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:25.687+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:25.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:25.703+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:26.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.031+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:26.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:26.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:26.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:26.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:26.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.059+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:26.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.243+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:26.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:26.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:26.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:26.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:26.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.283+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:26.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.283+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:26.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.283+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:26.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:26.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:26.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:26.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.711+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:26.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.723+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:26.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.727+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:02:26.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:26.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:26.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:26.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:26.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:26.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:26.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:26.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.851+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:26.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:26.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:27.183 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:27.183+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:02:27.207 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:27.207+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:02:27.228 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:27.227+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:27.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.327+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:27.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:27.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:27.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.379+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:27.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.391+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:02:27.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.403+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:27.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:27.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:27.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:27.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:29.251 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:02:29.252 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:02:29.252 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:02:29.367 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:02:29.367+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:02:29.368 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:02:29.392 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:02:30.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:30.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:30.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:30.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:30.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:30.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:30.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:30.191+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:31.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:31.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:32.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:32.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:32.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:32.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.099+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:32.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:32.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:32.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:32.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.275+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:32.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.415+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:32.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:32.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:32.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:32.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.483+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:32.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:32.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:32.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:32.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.643+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:32.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.687+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:32.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:32.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:32.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:32.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:32.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:32.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:32.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:32.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:32.795+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:32.898 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:32.895+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:02:35.425 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:35.419+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:02:35.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:35.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:35.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:35.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:35.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.667+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:35.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.667+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:35.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.695+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:35.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.695+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:35.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:35.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:35.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:35.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:35.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:35.735+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:36.839 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:36.839+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:37.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.191+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:37.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:37.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:37.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:37.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:37.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:37.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:37.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:37.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.227+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:37.403 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:37.399+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:02:37.425 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:37.423+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:02:37.471 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:37.471+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:02:37.472 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:37.471+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:37.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:37.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:37.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:37.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.523+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:37.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.523+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:37.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:37.527+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:39.405 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:02:39.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:02:39.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:02:39.511 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:02:39.511+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:02:39.513 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:02:39.541 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:02:39.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:39.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:39.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:39.923+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:39.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:39.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:39.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:39.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:39.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:39.987+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:39.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:39.987+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:39.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:39.991+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:43.414 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:43.411+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:02:43.439 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:43.439+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:44.187 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:44.187+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:02:44.188 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:44.187+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:44.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.251+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:44.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:44.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:44.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:44.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:44.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:44.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.291+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:44.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:44.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:47.746 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:47.738+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:48.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:48.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:48.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:48.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:48.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.238+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:48.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.238+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:48.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:48.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:48.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:48.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:48.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.338+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:48.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.338+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:48.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.354+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:48.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.966+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:02:48.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.966+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:02:48.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:49.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:48.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:49.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:49.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:49.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:49.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:49.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:49.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:49.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.378+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:02:49.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:49.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:49.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:49.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:49.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:49.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.402+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:49.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.402+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:49.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.402+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:49.542 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:02:49.542 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:02:49.543 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:02:49.724 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:02:49.722+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:02:49.725 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:02:49.738 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:02:49.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:49.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:49.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:49.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:49.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.922+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:49.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.922+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:49.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:49.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:53.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:53.158+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:53.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:53.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:53.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:53.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:53.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:53.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:53.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:53.210+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:54.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:54.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:54.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:54.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:54.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.198+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:54.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.198+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:54.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.226+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:54.447 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:54.446+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:02:54.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:54.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:54.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:54.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:54.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:54.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.866+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:54.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.874+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:54.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:54.882+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:55.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:55.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:55.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:55.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:55.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:55.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:55.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:55.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:55.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:55.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:55.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:55.346+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:55.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:55.362+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:56.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:02:56.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:02:56.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:02:56.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:02:56.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:02:56.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.274+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:02:56.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.274+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:02:56.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:02:56.274+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:02:58.855 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:58.854+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:58.866 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:58.866+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:02:58.867 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:58.866+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:02:59.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:02:59.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:02:59.740 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:02:59.885 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:02:59.882+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:02:59.886 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:02:59.908 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:02:59.997 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:02:59.994+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:03:00.037 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:00.034+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:00.041 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:00.038+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:03.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:03.198+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:03.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:03.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:03.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:03.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:03.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:03.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:03.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:03.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:03.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:03.210+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:03.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:03.910+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:04.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:04.466+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:05.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.510+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:05.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:05.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:05.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:05.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:05.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:05.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:05.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:05.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:05.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:05.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:05.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.570+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:05.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:05.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:05.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:05.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:05.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:05.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.734+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:05.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.734+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:05.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:05.734+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:06.312 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:06.310+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:08.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.910+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:08.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:08.934 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:08.934+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:03:08.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:08.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:08.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:08.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:08.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.950+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:08.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.950+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:08.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:08.950+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:08.953 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:08.950+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:09.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:09.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:09.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:09.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:09.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:09.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:09.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:09.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:09.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:09.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:09.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:09.846+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:09.909 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:03:09.909 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:03:09.909 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:03:09.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:09.946+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:10.050 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:10.046+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:10.098 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:03:10.094+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:03:10.098 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:03:10.103 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:03:10.114 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:10.110+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:10.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.242+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:10.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:10.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:10.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:10.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:10.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:10.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.310+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:10.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.310+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:10.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:10.310+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:11.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:11.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:11.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:11.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:11.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:11.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:11.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:11.870+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:12.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:12.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:12.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:12.194+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:12.935 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:12.934+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:03:12.936 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:12.934+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:03:14.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.558+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:03:14.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.566+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:03:14.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.586+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:14.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:14.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:14.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:14.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:14.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:14.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.614+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:14.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.614+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:14.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:14.614+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:16.605 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:16.602+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:03:16.606 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:16.602+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:03:18.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.082+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:18.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:18.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:18.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:18.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:18.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:18.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.118+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:18.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.118+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:18.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.118+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:18.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.206+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:18.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:18.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:18.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:18.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:18.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:18.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.242+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:18.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.242+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:18.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.242+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:18.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:18.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:18.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:18.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:18.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.334+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:18.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.334+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:18.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:18.350+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:19.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:19.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:19.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:19.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:19.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.262+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:19.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.278+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:19.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.550+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:03:19.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.550+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:03:19.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.554+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:19.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.554+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:19.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:19.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:19.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:19.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:19.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.594+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:19.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.594+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:19.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.598+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:19.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.918+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:19.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:19.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:19.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:19.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:19.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:19.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:20.117 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:03:20.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:03:20.118 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:03:20.246 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:03:20.242+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:03:20.246 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:03:20.257 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:03:20.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:20.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:20.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:20.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:20.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:20.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.942+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:20.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.946+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:20.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:20.946+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:22.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:22.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:22.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:22.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:22.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:22.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:22.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:22.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.114+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:22.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:22.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.274+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:22.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:22.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:22.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:22.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:22.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.294+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:22.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.294+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:22.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:22.294+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:24.804 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:24.802+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:26.125 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:26.122+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:03:26.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:26.166+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:26.173 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:26.170+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:27.445 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:27.442+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:27.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:27.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:27.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:27.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:27.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:27.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:27.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:27.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:27.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:27.550+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:27.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:27.550+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:30.079 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:30.077+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:30.094 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:30.089+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:30.259 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:03:30.260 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:03:30.260 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:03:30.452 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:03:30.449+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:03:30.453 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:03:30.478 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:03:30.833 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:30.829+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:03:30.834 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:30.829+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:33.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.125+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:33.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:33.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:33.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:33.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:33.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.173+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:33.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.173+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:33.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:33.173+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:34.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:34.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:34.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:34.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:34.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:34.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:34.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:34.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:34.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:34.681+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:34.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:34.681+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:34.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:34.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:36.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.009+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:36.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:36.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:36.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:36.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:36.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.053+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:36.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.053+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:36.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:36.069+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:36.131 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:36.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:37.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.061+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:37.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:37.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:37.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:37.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:37.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.117+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:37.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.117+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:37.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.117+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:37.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:37.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:37.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:37.617+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:38.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:38.017+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:38.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:38.017+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:38.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:38.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:38.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:38.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:38.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:38.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:38.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:38.061+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:39.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:39.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:39.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:39.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:39.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.077+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:39.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.077+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:39.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.089+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:39.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.093+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:03:39.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.093+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:39.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:39.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:39.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:39.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:39.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:39.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.129+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:39.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.129+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:39.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.149+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:39.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.405+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:39.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:39.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:39.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:39.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.449+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:39.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.449+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:39.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.449+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:39.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.709+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:03:39.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.713+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:03:39.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.721+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:39.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:39.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:39.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:39.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:39.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:39.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.769+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:39.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.769+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:39.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.777+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:39.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:39.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:39.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:39.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:39.893+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:40.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:40.033+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:40.480 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:03:40.481 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:03:40.481 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:03:40.584 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:03:40.581+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:03:40.585 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:03:40.610 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:03:41.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:41.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:41.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.273+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:41.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:41.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:41.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:41.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:41.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.801+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:41.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.801+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:41.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:41.801+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:42.120 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:42.117+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:42.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:42.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:42.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:42.739 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:42.737+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:03:42.757 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:42.753+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:42.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:42.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:42.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.793+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:42.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.793+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:42.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:42.793+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:44.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.093+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:03:44.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.093+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:03:44.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.101+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:44.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:44.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:44.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:44.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:44.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:44.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.125+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:44.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.125+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:44.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:44.125+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:45.240 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:45.237+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:03:45.259 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:45.257+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:45.259 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:45.257+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:45.564 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:45.561+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:45.606 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:45.601+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:46.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:46.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:46.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:46.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:46.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:46.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.105+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:46.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:46.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:46.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:46.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:46.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:46.169+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:46.986 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:46.985+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:47.041 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:47.037+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:49.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:49.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:49.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:49.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.413+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:49.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:49.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:49.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:49.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.937+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:49.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.937+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:03:49.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.937+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:03:49.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.953+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:49.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:49.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:49.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:49.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:49.981+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:50.113 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:50.109+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:03:50.120 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:50.109+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:03:50.121 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:50.117+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:50.131 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:50.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:50.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:03:50.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:03:50.611 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:03:50.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:50.713+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:50.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:50.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:50.735 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:03:50.733+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:03:50.736 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:03:50.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:50.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:50.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:50.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:50.747 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:03:50.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:50.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:50.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:50.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:50.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:50.749+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:51.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:51.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:51.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:51.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.149+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:51.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.149+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:51.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.149+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:51.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.221+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:51.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:51.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.281+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:51.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.513+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:51.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:51.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:51.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:51.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:51.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:51.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.593+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:51.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.593+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:51.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:51.609+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:52.425 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:52.421+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:03:52.448 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:52.445+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:53.522 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:53.517+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:03:53.522 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:53.517+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:03:53.523 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:53.517+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T01:03:53.523 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:53.517+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aff790 2023-10-14T01:03:53.523 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:53.517+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69b73ba0 2023-10-14T01:03:53.782 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:53.781+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:54.961 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:54.957+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:03:55.008 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:55.005+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:03:55.022 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:55.013+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:03:55.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.385+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:55.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:55.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:55.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:55.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:55.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:55.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.421+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:55.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.421+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:55.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.453+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:55.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.769+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:55.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:55.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:55.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:55.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.817+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:55.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.817+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:55.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:55.817+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:56.051 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:56.049+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:03:56.052 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:03:56.049+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:03:56.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.169+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:03:56.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:56.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:56.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:56.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:56.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:56.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.213+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:56.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.213+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:56.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.225+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:56.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:03:56.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:03:56.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:03:56.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:03:56.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:03:56.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.937+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:03:56.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.937+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:03:56.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:03:56.945+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:03:57.237 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:03:57.233+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:00.750 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:04:00.750 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:04:00.750 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:04:00.864 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:04:00.861+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:04:00.865 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:04:00.894 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:04:01.380 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:01.377+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:04:01.426 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:01.425+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:01.427 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:01.425+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:01.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:01.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:01.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:01.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:01.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:01.925+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:01.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:01.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:02.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:02.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:02.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:02.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:02.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:02.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:02.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:02.045+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:04.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.645+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:04.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:04.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.705+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:04.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:04.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:04.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.753+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:04.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.753+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:04.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:04.753+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:05.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:05.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:05.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:05.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:05.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:05.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:06.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:05.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:06.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:05.989+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:06.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:05.989+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:06.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:05.989+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:06.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:06.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:06.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:06.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:06.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:06.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:06.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:06.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:06.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:06.225+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:06.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:06.245+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:06.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:06.245+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:07.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:07.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:07.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:07.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.057+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:07.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.057+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:07.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.057+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:07.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:07.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:07.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:07.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:07.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:07.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.365+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:07.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.365+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:07.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:07.365+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:09.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.617+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:09.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:09.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:09.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:09.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:09.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.665+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:09.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.665+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:09.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:09.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:09.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:09.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:09.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:09.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.913+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:09.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.913+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:09.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:09.913+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:10.899 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:04:10.899 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:04:10.899 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:04:11.020 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:04:11.022 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:04:11.016+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:04:11.043 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:04:12.097 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:12.092+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:12.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:12.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:12.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:12.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:12.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:12.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:12.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:12.148+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:12.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:12.148+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:12.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:12.168+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:14.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:14.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:14.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:14.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:14.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:14.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:14.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:14.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:14.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:14.776+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:14.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:14.776+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:14.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:14.776+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:16.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:16.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:16.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:16.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:16.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:16.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:16.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:16.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:16.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:16.972+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:16.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:16.972+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:16.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:16.976+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:17.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:17.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:17.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:17.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:17.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:17.328+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:17.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:17.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:17.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:17.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:17.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:17.448+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:18.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:18.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:18.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:18.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:18.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:18.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.372+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:18.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.372+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:18.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:18.384+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:19.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:19.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:19.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:19.176+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:19.933 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:19.928+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:04:19.956 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:19.948+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:19.956 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:19.952+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:21.051 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:04:21.051 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:04:21.052 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:04:21.162 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:04:21.156+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:04:21.163 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:04:21.170 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:04:23.153 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:23.148+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:24.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.060+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:24.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:24.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:24.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:24.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:24.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:24.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.224+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:24.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.224+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:24.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.224+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:24.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.452+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:24.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:24.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:24.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:24.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:24.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:24.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:24.488+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:25.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.100+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:25.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:25.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:25.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:25.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:25.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:25.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.128+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:25.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.128+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:25.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.140+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:25.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.872+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:25.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:25.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:25.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:25.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:25.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:25.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.920+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:25.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.920+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:25.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:25.936+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:26.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:26.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:26.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:26.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.560+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:26.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.560+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:26.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.596+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:26.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.704+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:26.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:26.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:26.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:26.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:27.242 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:27.236+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:29.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.632+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:29.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:29.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:29.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:29.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:29.696+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:29.940 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:29.932+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:29.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:29.940+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:31.171 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:04:31.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:04:31.172 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:04:31.351 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:04:31.348+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:04:31.352 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:04:31.371 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:04:31.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:31.456+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:31.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:31.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:31.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:31.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:31.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:31.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:31.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:31.508+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:34.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.232+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:34.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:34.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:34.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:34.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:34.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:34.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:34.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.340+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:34.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:34.344+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:35.182 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:35.176+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:35.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:35.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:35.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:35.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:35.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:35.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:35.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:35.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:35.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:35.736+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:37.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.640+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:37.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:37.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:37.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:37.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:37.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:37.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.692+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:37.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:37.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.832+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:37.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:37.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:37.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:37.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:37.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:37.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.860+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:37.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.860+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:37.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:37.860+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:38.035 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:38.032+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:04:38.036 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:38.032+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:38.038 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:38.032+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:38.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:38.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:38.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:38.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:38.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.268+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:38.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.556+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:38.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:38.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:38.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:38.600+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:39.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.072+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:39.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:39.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:39.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:39.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:39.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:39.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.156+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:39.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.156+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:39.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:39.156+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:39.187 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:39.184+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:39.757 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:39.752+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:40.199 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:40.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:04:40.215 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:40.212+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:40.230 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:40.228+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:40.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:40.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:40.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:40.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:40.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:40.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:40.854 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:40.848+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:41.230 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:41.228+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:41.232 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:41.228+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:41.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:04:41.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:04:41.406 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:04:41.606 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:04:41.604+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:04:41.607 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:04:41.617 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:04:41.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.908+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:41.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:41.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:41.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:41.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:41.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:41.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.972+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:41.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.972+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:41.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:41.996+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:42.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:42.032 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:42.032+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:04:42.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:42.065 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:42.064+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:42.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:42.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:42.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.064+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:42.077 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:42.072+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:42.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:42.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:42.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:42.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:42.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:42.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.900+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:42.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:42.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:43.930 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:43.928+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:45.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:45.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:45.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:45.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:45.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:45.216+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:46.480 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:46.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:46.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:46.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:46.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:46.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:46.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:46.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:46.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:46.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:46.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:46.640+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:46.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:46.640+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:46.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:46.640+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:48.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:48.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:48.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:48.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:48.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:48.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:48.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:48.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:48.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:48.588+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:50.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:50.544+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:50.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:50.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:50.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:50.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:50.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:50.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:50.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:50.632+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:50.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:50.632+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:50.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:50.636+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:51.044 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:51.044+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:04:51.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.144+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:51.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:51.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:51.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:51.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:51.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:51.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.216+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:51.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.216+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:51.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.236+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:51.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.484+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:51.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:51.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:51.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:51.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:51.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:51.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.508+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:51.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.508+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:51.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:51.512+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:51.618 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:04:51.618 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:04:51.618 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:04:51.773 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:04:51.772+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:04:51.782 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:04:51.795 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:04:54.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.103+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:54.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:54.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:54.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:04:54.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:04:54.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.139+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:04:54.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.139+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:04:54.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:54.139+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:55.509 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:55.507+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:04:55.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:55.691+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:04:55.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:55.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:04:55.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:55.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:04:55.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:55.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:04:55.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:04:55.739+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:04:59.643 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:59.643+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:04:59.668 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:04:59.667+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:01.796 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:05:01.797 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:05:01.797 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:05:01.966 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:05:01.963+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:05:01.966 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:05:01.986 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:05:02.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:02.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:02.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:02.887+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:02.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:02.899+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:02.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:02.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:02.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:02.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:02.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:02.919+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:02.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:02.943+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:06.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.071+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:06.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:06.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:06.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:06.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:06.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.087+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:06.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.087+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:06.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.087+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:06.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.319+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:06.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:06.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:06.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:06.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:06.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.339+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:06.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.339+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:06.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.343+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:06.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:06.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:06.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:06.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:06.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.371+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:06.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.371+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:06.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.387+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:06.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:06.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:08.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:08.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:08.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:08.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:08.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:08.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:08.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:08.119+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:09.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.199+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:09.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:09.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:09.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:09.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:09.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:09.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.319+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:09.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.319+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:09.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:09.375+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:10.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.227+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:10.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:10.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:10.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:10.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:10.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:10.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.247+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:10.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.247+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:10.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:10.247+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:11.041 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:11.039+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:11.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:11.763+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:11.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:11.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:11.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:11.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:11.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:11.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:11.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:11.815+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:11.989 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:05:11.989 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:05:11.989 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule add -p rbd --image image1 2m 2023-10-14T01:05:12.117 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:05:12.115+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:05:12.118 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule add failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:05:12.137 INFO:tasks.workunit.client.0.smithi183.stderr:+ sleep 10 2023-10-14T01:05:13.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:13.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:13.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:13.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:13.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.787+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:13.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.787+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:13.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.787+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:13.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:13.827+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:15.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:15.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:15.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:15.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:15.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:15.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:15.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:15.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:15.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:15.351+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:15.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:15.351+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:15.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:15.371+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:15.383 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:15.383+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:05:15.400 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:15.399+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:05:15.405 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:15.403+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:05:15.405 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:15.403+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:16.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:16.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:16.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:16.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.359+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:16.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.359+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:16.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.363+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:16.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:16.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:16.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:16.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.411+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:16.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.411+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:16.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:16.415+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:19.032 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:19.031+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:05:19.071 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:19.071+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:19.501 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:19.499+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:05:19.519 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:19.519+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:20.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:20.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:20.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:20.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:20.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:20.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:20.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:20.235+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:20.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:20.235+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:20.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:20.235+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:20.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:20.251+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:21.301 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:05:21.299+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:05:21.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.815+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:21.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:21.833 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:21.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:21.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:21.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.839+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:21.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:21.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:21.923+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:22.138 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i++ )) 2023-10-14T01:05:22.138 INFO:tasks.workunit.client.0.smithi183.stderr:+ (( i <= 24 )) 2023-10-14T01:05:22.139 INFO:tasks.workunit.client.0.smithi183.stderr:+ grep 'every 2m' 2023-10-14T01:05:22.142 INFO:tasks.workunit.client.0.smithi183.stderr:+ rbd mirror snapshot schedule ls -p rbd --image image1 2023-10-14T01:05:22.277 INFO:tasks.ceph.mgr.x.smithi183.stderr:2023-10-14T01:05:22.275+0000 7f66099e6640 -1 mgr.server reply reply (11) Resource temporarily unavailable rbd_support module is not ready, try again 2023-10-14T01:05:22.278 INFO:tasks.workunit.client.0.smithi183.stderr:rbd: rbd mirror snapshot schedule list failed: (11) Resource temporarily unavailable: rbd_support module is not ready, try again 2023-10-14T01:05:22.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T01:05:22.409 INFO:tasks.workunit:Stopping ['rbd/rbd_support_module_recovery.sh'] on client.0... 2023-10-14T01:05:22.410 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-10-14T01:05:22.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.527+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:22.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:22.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:22.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:22.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:22.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.543+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:22.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.555+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:22.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:22.567+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:23.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:23.039+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:23.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:23.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:23.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:23.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:23.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:23.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:23.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:23.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:23.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:23.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:23.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:23.079+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:23.238 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/rbd_support_module_recovery.sh) on smithi183 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=735f68d7bf91d643813a8420fc2b11441f4bd8c4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd_support_module_recovery.sh' 2023-10-14T01:05:23.322 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=146b1c4b7a81412e8ebb7743359e5ed9 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa/tasks/workunit.py", line 129, in task p.spawn(_run_tests, ctx, refspec, role, tests, File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_idryomov_ceph_735f68d7bf91d643813a8420fc2b11441f4bd8c4/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test rbd/rbd_support_module_recovery.sh) on smithi183 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=735f68d7bf91d643813a8420fc2b11441f4bd8c4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd_support_module_recovery.sh' 2023-10-14T01:05:23.330 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-14T01:05:23.382 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-14T01:05:23.391 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-14T01:05:23.392 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:05:24.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.275+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:24.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:24.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:24.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:24.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:24.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.315+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:24.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.315+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:24.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:24.315+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:24.689 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:05:24.690 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:05:24.720 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2091,"stamp":"2023-10-14T01:05:24.542900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26849963596,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1927523,"num_read_kb":7484435,"num_write":1891551,"num_write_kb":7565993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80463,"ondisk_log_size":80463,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19248416,"kb_used_data":15130752,"kb_used_omap":1902,"kb_used_meta":4115729,"kb_avail":263867104,"statfs":{"total":289910292480,"available":270199914496,"internally_reserved":0,"allocated":15493890048,"data_stored":25369222754,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1947695,"internal_metadata":4214507473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,16,7,8,7,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18747392,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4337,"num_read_kb":17312,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009832"},"pg_stats":[{"pgid":"2.7","version":"118'222112","reported_seq":442005,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.527763+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:05:21.527763+0000","last_peered":"2023-10-14T01:05:21.527763+0000","last_clean":"2023-10-14T01:05:21.527763+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:05:21.527763+0000","last_undegraded":"2023-10-14T01:05:21.527763+0000","last_fullsized":"2023-10-14T01:05:21.527763+0000","mapping_epoch":13,"log_start":"116'212111","ondisk_log_start":"116'212111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:25:08.590095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3125895351,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":220767,"num_read_kb":868612,"num_write":221056,"num_write_kb":884209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"118'232879","reported_seq":460857,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.547309+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:05:21.547309+0000","last_peered":"2023-10-14T01:05:21.547309+0000","last_clean":"2023-10-14T01:05:21.547309+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:05:21.547309+0000","last_undegraded":"2023-10-14T01:05:21.547309+0000","last_fullsized":"2023-10-14T01:05:21.547309+0000","mapping_epoch":13,"log_start":"116'222808","ondisk_log_start":"116'222808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:48:19.383773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3149398283,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":230718,"num_read_kb":885869,"num_write":221252,"num_write_kb":884607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"118'238151","reported_seq":473191,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.580326+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:05:21.580326+0000","last_peered":"2023-10-14T01:05:21.580326+0000","last_clean":"2023-10-14T01:05:21.580326+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:05:21.580326+0000","last_undegraded":"2023-10-14T01:05:21.580326+0000","last_fullsized":"2023-10-14T01:05:21.580326+0000","mapping_epoch":13,"log_start":"116'228112","ondisk_log_start":"116'228112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T07:59:21.530274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3312353835,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":236137,"num_read_kb":925414,"num_write":233472,"num_write_kb":933767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"118'251627","reported_seq":495307,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.573862+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:05:21.573862+0000","last_peered":"2023-10-14T01:05:21.573862+0000","last_clean":"2023-10-14T01:05:21.573862+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:05:21.573862+0000","last_undegraded":"2023-10-14T01:05:21.573862+0000","last_fullsized":"2023-10-14T01:05:21.573862+0000","mapping_epoch":13,"log_start":"116'241610","ondisk_log_start":"116'241610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:18:04.294509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3402772911,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":245247,"num_read_kb":948435,"num_write":239996,"num_write_kb":959666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'242613","reported_seq":485310,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:24.012628+0000","last_change":"2023-10-14T00:16:06.071834+0000","last_active":"2023-10-14T01:05:24.012628+0000","last_peered":"2023-10-14T01:05:24.012628+0000","last_clean":"2023-10-14T01:05:24.012628+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:05:24.012628+0000","last_undegraded":"2023-10-14T01:05:24.012628+0000","last_fullsized":"2023-10-14T01:05:24.012628+0000","mapping_epoch":13,"log_start":"116'232612","ondisk_log_start":"116'232612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:53:17.523335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3355378135,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":244133,"num_read_kb":936895,"num_write":237248,"num_write_kb":948485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"118'254061","reported_seq":505988,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:20.391895+0000","last_change":"2023-10-14T00:16:06.066447+0000","last_active":"2023-10-14T01:05:20.391895+0000","last_peered":"2023-10-14T01:05:20.391895+0000","last_clean":"2023-10-14T01:05:20.391895+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:05:20.391895+0000","last_undegraded":"2023-10-14T01:05:20.391895+0000","last_fullsized":"2023-10-14T01:05:20.391895+0000","mapping_epoch":13,"log_start":"116'244011","ondisk_log_start":"116'244011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:15:31.599191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3560255908,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":253107,"num_read_kb":987436,"num_write":249562,"num_write_kb":998074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"118'251573","reported_seq":499561,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:20.422737+0000","last_change":"2023-10-14T00:16:06.066485+0000","last_active":"2023-10-14T01:05:20.422737+0000","last_peered":"2023-10-14T01:05:20.422737+0000","last_clean":"2023-10-14T01:05:20.422737+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:05:20.422737+0000","last_undegraded":"2023-10-14T01:05:20.422737+0000","last_fullsized":"2023-10-14T01:05:20.422737+0000","mapping_epoch":13,"log_start":"116'241513","ondisk_log_start":"116'241513","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:40:19.002518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3511865697,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":249225,"num_read_kb":972594,"num_write":247068,"num_write_kb":988082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"118'256813","reported_seq":503276,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.572219+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:05:21.572219+0000","last_peered":"2023-10-14T01:05:21.572219+0000","last_clean":"2023-10-14T01:05:21.572219+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:05:21.572219+0000","last_undegraded":"2023-10-14T01:05:21.572219+0000","last_fullsized":"2023-10-14T01:05:21.572219+0000","mapping_epoch":13,"log_start":"116'246714","ondisk_log_start":"116'246714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T04:01:06.210461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3431453108,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":248063,"num_read_kb":959071,"num_write":241728,"num_write_kb":966555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"53'125","reported_seq":309,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:04:13.280454+0000","last_change":"2023-10-13T23:59:38.935251+0000","last_active":"2023-10-14T01:04:13.280454+0000","last_peered":"2023-10-14T01:04:13.280454+0000","last_clean":"2023-10-14T01:04:13.280454+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:04:13.280454+0000","last_undegraded":"2023-10-14T01:04:13.280454+0000","last_fullsized":"2023-10-14T01:04:13.280454+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_clean_scrub_stamp":"2023-10-13T23:59:37.743218+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:21:33.168386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":26849373228,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1927397,"num_read_kb":7484326,"num_write":1891382,"num_write_kb":7563445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15491772416,"data_stored":25367857240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80338,"ondisk_log_size":80338,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739155,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3954344,"kb_used_data":2953120,"kb_used_omap":349,"kb_used_meta":1000866,"kb_avail":90417496,"statfs":{"total":96636764160,"available":92587515904,"internally_reserved":0,"allocated":3023994880,"data_stored":4952395678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1024887121},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,8,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.82199999999999995,"15min":0.75900000000000001},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":3.7749999999999999,"5min":16.474,"15min":16.474},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.70599999999999996,"15min":0.83299999999999996},"min":{"1min":0.21099999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.88700000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.20599999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":1.641,"15min":1.845},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":13.519,"5min":13.519,"15min":14.303000000000001},"last":8.3849999999999998},{"interface":"front","average":{"1min":2.2389999999999999,"5min":1.6870000000000001,"15min":1.905},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":13.565,"5min":13.565,"15min":14.35},"last":8.4800000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739148,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9484180,"kb_used_data":7563980,"kb_used_omap":959,"kb_used_meta":1919232,"kb_avail":84887660,"statfs":{"total":96636764160,"available":86924963840,"internally_reserved":0,"allocated":7745515520,"data_stored":12683487537,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":983010,"internal_metadata":1965293598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,6,4,6,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":1.0629999999999999,"15min":1.0029999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":4.2380000000000004,"5min":9.548,"15min":14.773},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.70899999999999996,"15min":0.95799999999999996},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.20799999999999999},"max":{"1min":1.847,"5min":9.593,"15min":10.161},"last":0.27400000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.68799999999999994,"15min":0.875},"min":{"1min":0.27400000000000002,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":1.677,"5min":6.7370000000000001,"15min":9.5099999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.056,"5min":0.90300000000000002,"15min":0.78700000000000003},"min":{"1min":0.20200000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":4.3940000000000001,"5min":9.2200000000000006,"15min":9.2200000000000006},"last":0.375}]}]},{"osd":0,"up_from":8,"seq":34359739156,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5809892,"kb_used_data":4613652,"kb_used_omap":592,"kb_used_meta":1195631,"kb_avail":88561948,"statfs":{"total":96636764160,"available":90687434752,"internally_reserved":0,"allocated":4724379648,"data_stored":7733339539,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1224326754},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.72299999999999998},"min":{"1min":0.19900000000000001,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":4.2210000000000001,"5min":6.6829999999999998,"15min":6.7480000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.99199999999999999,"15min":1.0840000000000001},"min":{"1min":0.27000000000000002,"5min":0.191,"15min":0.191},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":9.077},"last":9.4849999999999994}]},{"osd":2,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.218,"15min":1.1759999999999999},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":12.622999999999999,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0269999999999999,"15min":0.98399999999999999},"min":{"1min":0.20999999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":6.319,"5min":8.0009999999999994,"15min":9.1259999999999994},"last":9.5310000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4723507200,"data_stored":7732545395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7744946176,"data_stored":12682988588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3023319040,"data_stored":4952323257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:05:24.721 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:05:25.430 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:05:25.431 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:05:25.500 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2091,"stamp":"2023-10-14T01:05:24.542900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26849963596,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1927523,"num_read_kb":7484435,"num_write":1891551,"num_write_kb":7565993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80463,"ondisk_log_size":80463,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19248416,"kb_used_data":15130752,"kb_used_omap":1902,"kb_used_meta":4115729,"kb_avail":263867104,"statfs":{"total":289910292480,"available":270199914496,"internally_reserved":0,"allocated":15493890048,"data_stored":25369222754,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1947695,"internal_metadata":4214507473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,16,7,8,7,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18747392,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4337,"num_read_kb":17312,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009832"},"pg_stats":[{"pgid":"2.7","version":"118'222112","reported_seq":442005,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.527763+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:05:21.527763+0000","last_peered":"2023-10-14T01:05:21.527763+0000","last_clean":"2023-10-14T01:05:21.527763+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:05:21.527763+0000","last_undegraded":"2023-10-14T01:05:21.527763+0000","last_fullsized":"2023-10-14T01:05:21.527763+0000","mapping_epoch":13,"log_start":"116'212111","ondisk_log_start":"116'212111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:25:08.590095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3125895351,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":220767,"num_read_kb":868612,"num_write":221056,"num_write_kb":884209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"118'232879","reported_seq":460857,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.547309+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:05:21.547309+0000","last_peered":"2023-10-14T01:05:21.547309+0000","last_clean":"2023-10-14T01:05:21.547309+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:05:21.547309+0000","last_undegraded":"2023-10-14T01:05:21.547309+0000","last_fullsized":"2023-10-14T01:05:21.547309+0000","mapping_epoch":13,"log_start":"116'222808","ondisk_log_start":"116'222808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:48:19.383773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3149398283,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":230718,"num_read_kb":885869,"num_write":221252,"num_write_kb":884607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"118'238151","reported_seq":473191,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.580326+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:05:21.580326+0000","last_peered":"2023-10-14T01:05:21.580326+0000","last_clean":"2023-10-14T01:05:21.580326+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:05:21.580326+0000","last_undegraded":"2023-10-14T01:05:21.580326+0000","last_fullsized":"2023-10-14T01:05:21.580326+0000","mapping_epoch":13,"log_start":"116'228112","ondisk_log_start":"116'228112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T07:59:21.530274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3312353835,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":236137,"num_read_kb":925414,"num_write":233472,"num_write_kb":933767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"118'251627","reported_seq":495307,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.573862+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:05:21.573862+0000","last_peered":"2023-10-14T01:05:21.573862+0000","last_clean":"2023-10-14T01:05:21.573862+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:05:21.573862+0000","last_undegraded":"2023-10-14T01:05:21.573862+0000","last_fullsized":"2023-10-14T01:05:21.573862+0000","mapping_epoch":13,"log_start":"116'241610","ondisk_log_start":"116'241610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:18:04.294509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3402772911,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":245247,"num_read_kb":948435,"num_write":239996,"num_write_kb":959666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'242613","reported_seq":485310,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:24.012628+0000","last_change":"2023-10-14T00:16:06.071834+0000","last_active":"2023-10-14T01:05:24.012628+0000","last_peered":"2023-10-14T01:05:24.012628+0000","last_clean":"2023-10-14T01:05:24.012628+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:05:24.012628+0000","last_undegraded":"2023-10-14T01:05:24.012628+0000","last_fullsized":"2023-10-14T01:05:24.012628+0000","mapping_epoch":13,"log_start":"116'232612","ondisk_log_start":"116'232612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:53:17.523335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3355378135,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":244133,"num_read_kb":936895,"num_write":237248,"num_write_kb":948485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"118'254061","reported_seq":505988,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:20.391895+0000","last_change":"2023-10-14T00:16:06.066447+0000","last_active":"2023-10-14T01:05:20.391895+0000","last_peered":"2023-10-14T01:05:20.391895+0000","last_clean":"2023-10-14T01:05:20.391895+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:05:20.391895+0000","last_undegraded":"2023-10-14T01:05:20.391895+0000","last_fullsized":"2023-10-14T01:05:20.391895+0000","mapping_epoch":13,"log_start":"116'244011","ondisk_log_start":"116'244011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:15:31.599191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3560255908,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":253107,"num_read_kb":987436,"num_write":249562,"num_write_kb":998074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"118'251573","reported_seq":499561,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:20.422737+0000","last_change":"2023-10-14T00:16:06.066485+0000","last_active":"2023-10-14T01:05:20.422737+0000","last_peered":"2023-10-14T01:05:20.422737+0000","last_clean":"2023-10-14T01:05:20.422737+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:05:20.422737+0000","last_undegraded":"2023-10-14T01:05:20.422737+0000","last_fullsized":"2023-10-14T01:05:20.422737+0000","mapping_epoch":13,"log_start":"116'241513","ondisk_log_start":"116'241513","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:40:19.002518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3511865697,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":249225,"num_read_kb":972594,"num_write":247068,"num_write_kb":988082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"118'256813","reported_seq":503276,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.572219+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:05:21.572219+0000","last_peered":"2023-10-14T01:05:21.572219+0000","last_clean":"2023-10-14T01:05:21.572219+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:05:21.572219+0000","last_undegraded":"2023-10-14T01:05:21.572219+0000","last_fullsized":"2023-10-14T01:05:21.572219+0000","mapping_epoch":13,"log_start":"116'246714","ondisk_log_start":"116'246714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T04:01:06.210461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3431453108,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":248063,"num_read_kb":959071,"num_write":241728,"num_write_kb":966555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"53'125","reported_seq":309,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:04:13.280454+0000","last_change":"2023-10-13T23:59:38.935251+0000","last_active":"2023-10-14T01:04:13.280454+0000","last_peered":"2023-10-14T01:04:13.280454+0000","last_clean":"2023-10-14T01:04:13.280454+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:04:13.280454+0000","last_undegraded":"2023-10-14T01:04:13.280454+0000","last_fullsized":"2023-10-14T01:04:13.280454+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_clean_scrub_stamp":"2023-10-13T23:59:37.743218+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:21:33.168386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":26849373228,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1927397,"num_read_kb":7484326,"num_write":1891382,"num_write_kb":7563445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15491772416,"data_stored":25367857240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80338,"ondisk_log_size":80338,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739155,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3954344,"kb_used_data":2953120,"kb_used_omap":349,"kb_used_meta":1000866,"kb_avail":90417496,"statfs":{"total":96636764160,"available":92587515904,"internally_reserved":0,"allocated":3023994880,"data_stored":4952395678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1024887121},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,8,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.82199999999999995,"15min":0.75900000000000001},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":3.7749999999999999,"5min":16.474,"15min":16.474},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.70599999999999996,"15min":0.83299999999999996},"min":{"1min":0.21099999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.88700000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.20599999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":1.641,"15min":1.845},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":13.519,"5min":13.519,"15min":14.303000000000001},"last":8.3849999999999998},{"interface":"front","average":{"1min":2.2389999999999999,"5min":1.6870000000000001,"15min":1.905},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":13.565,"5min":13.565,"15min":14.35},"last":8.4800000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739148,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9484180,"kb_used_data":7563980,"kb_used_omap":959,"kb_used_meta":1919232,"kb_avail":84887660,"statfs":{"total":96636764160,"available":86924963840,"internally_reserved":0,"allocated":7745515520,"data_stored":12683487537,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":983010,"internal_metadata":1965293598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,6,4,6,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":1.0629999999999999,"15min":1.0029999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":4.2380000000000004,"5min":9.548,"15min":14.773},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.70899999999999996,"15min":0.95799999999999996},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.20799999999999999},"max":{"1min":1.847,"5min":9.593,"15min":10.161},"last":0.27400000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.68799999999999994,"15min":0.875},"min":{"1min":0.27400000000000002,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":1.677,"5min":6.7370000000000001,"15min":9.5099999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.056,"5min":0.90300000000000002,"15min":0.78700000000000003},"min":{"1min":0.20200000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":4.3940000000000001,"5min":9.2200000000000006,"15min":9.2200000000000006},"last":0.375}]}]},{"osd":0,"up_from":8,"seq":34359739156,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5809892,"kb_used_data":4613652,"kb_used_omap":592,"kb_used_meta":1195631,"kb_avail":88561948,"statfs":{"total":96636764160,"available":90687434752,"internally_reserved":0,"allocated":4724379648,"data_stored":7733339539,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1224326754},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.72299999999999998},"min":{"1min":0.19900000000000001,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":4.2210000000000001,"5min":6.6829999999999998,"15min":6.7480000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.99199999999999999,"15min":1.0840000000000001},"min":{"1min":0.27000000000000002,"5min":0.191,"15min":0.191},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":9.077},"last":9.4849999999999994}]},{"osd":2,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.218,"15min":1.1759999999999999},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":12.622999999999999,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0269999999999999,"15min":0.98399999999999999},"min":{"1min":0.20999999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":6.319,"5min":8.0009999999999994,"15min":9.1259999999999994},"last":9.5310000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4723507200,"data_stored":7732545395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7744946176,"data_stored":12682988588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3023319040,"data_stored":4952323257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:05:25.502 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-14T01:05:25.502 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:05:25.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.891+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:25.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:25.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:25.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:25.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.943+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:25.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.943+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:25.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.943+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:25.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.943+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:25.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:25.967+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:26.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.075+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:05:26.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.075+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:26.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:26.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:26.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:26.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:26.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:26.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.103+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:26.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.103+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:26.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:26.111+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:05:26.171 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:05:26.244 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2091,"stamp":"2023-10-14T01:05:24.542900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26849963596,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1927523,"num_read_kb":7484435,"num_write":1891551,"num_write_kb":7565993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80463,"ondisk_log_size":80463,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19248416,"kb_used_data":15130752,"kb_used_omap":1902,"kb_used_meta":4115729,"kb_avail":263867104,"statfs":{"total":289910292480,"available":270199914496,"internally_reserved":0,"allocated":15493890048,"data_stored":25369222754,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1947695,"internal_metadata":4214507473},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,16,7,8,7,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18747392,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4337,"num_read_kb":17312,"num_write":4562,"num_write_kb":18248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.009832"},"pg_stats":[{"pgid":"2.7","version":"118'222112","reported_seq":442005,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.527763+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:05:21.527763+0000","last_peered":"2023-10-14T01:05:21.527763+0000","last_clean":"2023-10-14T01:05:21.527763+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:05:21.527763+0000","last_undegraded":"2023-10-14T01:05:21.527763+0000","last_fullsized":"2023-10-14T01:05:21.527763+0000","mapping_epoch":13,"log_start":"116'212111","ondisk_log_start":"116'212111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:25:08.590095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3125895351,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":220767,"num_read_kb":868612,"num_write":221056,"num_write_kb":884209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"118'232879","reported_seq":460857,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.547309+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:05:21.547309+0000","last_peered":"2023-10-14T01:05:21.547309+0000","last_clean":"2023-10-14T01:05:21.547309+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:05:21.547309+0000","last_undegraded":"2023-10-14T01:05:21.547309+0000","last_fullsized":"2023-10-14T01:05:21.547309+0000","mapping_epoch":13,"log_start":"116'222808","ondisk_log_start":"116'222808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:48:19.383773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3149398283,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":230718,"num_read_kb":885869,"num_write":221252,"num_write_kb":884607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"118'238151","reported_seq":473191,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.580326+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:05:21.580326+0000","last_peered":"2023-10-14T01:05:21.580326+0000","last_clean":"2023-10-14T01:05:21.580326+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:05:21.580326+0000","last_undegraded":"2023-10-14T01:05:21.580326+0000","last_fullsized":"2023-10-14T01:05:21.580326+0000","mapping_epoch":13,"log_start":"116'228112","ondisk_log_start":"116'228112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T07:59:21.530274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3312353835,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":236137,"num_read_kb":925414,"num_write":233472,"num_write_kb":933767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"118'251627","reported_seq":495307,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.573862+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:05:21.573862+0000","last_peered":"2023-10-14T01:05:21.573862+0000","last_clean":"2023-10-14T01:05:21.573862+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:05:21.573862+0000","last_undegraded":"2023-10-14T01:05:21.573862+0000","last_fullsized":"2023-10-14T01:05:21.573862+0000","mapping_epoch":13,"log_start":"116'241610","ondisk_log_start":"116'241610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:18:04.294509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3402772911,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":245247,"num_read_kb":948435,"num_write":239996,"num_write_kb":959666,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'242613","reported_seq":485310,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:24.012628+0000","last_change":"2023-10-14T00:16:06.071834+0000","last_active":"2023-10-14T01:05:24.012628+0000","last_peered":"2023-10-14T01:05:24.012628+0000","last_clean":"2023-10-14T01:05:24.012628+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:05:24.012628+0000","last_undegraded":"2023-10-14T01:05:24.012628+0000","last_fullsized":"2023-10-14T01:05:24.012628+0000","mapping_epoch":13,"log_start":"116'232612","ondisk_log_start":"116'232612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:53:17.523335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3355378135,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":244133,"num_read_kb":936895,"num_write":237248,"num_write_kb":948485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"118'254061","reported_seq":505988,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:20.391895+0000","last_change":"2023-10-14T00:16:06.066447+0000","last_active":"2023-10-14T01:05:20.391895+0000","last_peered":"2023-10-14T01:05:20.391895+0000","last_clean":"2023-10-14T01:05:20.391895+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:05:20.391895+0000","last_undegraded":"2023-10-14T01:05:20.391895+0000","last_fullsized":"2023-10-14T01:05:20.391895+0000","mapping_epoch":13,"log_start":"116'244011","ondisk_log_start":"116'244011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:15:31.599191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3560255908,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":253107,"num_read_kb":987436,"num_write":249562,"num_write_kb":998074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"118'251573","reported_seq":499561,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:20.422737+0000","last_change":"2023-10-14T00:16:06.066485+0000","last_active":"2023-10-14T01:05:20.422737+0000","last_peered":"2023-10-14T01:05:20.422737+0000","last_clean":"2023-10-14T01:05:20.422737+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:05:20.422737+0000","last_undegraded":"2023-10-14T01:05:20.422737+0000","last_fullsized":"2023-10-14T01:05:20.422737+0000","mapping_epoch":13,"log_start":"116'241513","ondisk_log_start":"116'241513","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:40:19.002518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3511865697,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":249225,"num_read_kb":972594,"num_write":247068,"num_write_kb":988082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"118'256813","reported_seq":503276,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:21.572219+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:05:21.572219+0000","last_peered":"2023-10-14T01:05:21.572219+0000","last_clean":"2023-10-14T01:05:21.572219+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:05:21.572219+0000","last_undegraded":"2023-10-14T01:05:21.572219+0000","last_fullsized":"2023-10-14T01:05:21.572219+0000","mapping_epoch":13,"log_start":"116'246714","ondisk_log_start":"116'246714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T04:01:06.210461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3431453108,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":248063,"num_read_kb":959071,"num_write":241728,"num_write_kb":966555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"53'125","reported_seq":309,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:04:13.280454+0000","last_change":"2023-10-13T23:59:38.935251+0000","last_active":"2023-10-14T01:04:13.280454+0000","last_peered":"2023-10-14T01:04:13.280454+0000","last_clean":"2023-10-14T01:04:13.280454+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:04:13.280454+0000","last_undegraded":"2023-10-14T01:04:13.280454+0000","last_fullsized":"2023-10-14T01:04:13.280454+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_clean_scrub_stamp":"2023-10-13T23:59:37.743218+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:21:33.168386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":26849373228,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1927397,"num_read_kb":7484326,"num_write":1891382,"num_write_kb":7563445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15491772416,"data_stored":25367857240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80338,"ondisk_log_size":80338,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739155,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3954344,"kb_used_data":2953120,"kb_used_omap":349,"kb_used_meta":1000866,"kb_avail":90417496,"statfs":{"total":96636764160,"available":92587515904,"internally_reserved":0,"allocated":3023994880,"data_stored":4952395678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1024887121},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,8,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.82199999999999995,"15min":0.75900000000000001},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":3.7749999999999999,"5min":16.474,"15min":16.474},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.70599999999999996,"15min":0.83299999999999996},"min":{"1min":0.21099999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.88700000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.20599999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":1.641,"15min":1.845},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":13.519,"5min":13.519,"15min":14.303000000000001},"last":8.3849999999999998},{"interface":"front","average":{"1min":2.2389999999999999,"5min":1.6870000000000001,"15min":1.905},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":13.565,"5min":13.565,"15min":14.35},"last":8.4800000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739148,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9484180,"kb_used_data":7563980,"kb_used_omap":959,"kb_used_meta":1919232,"kb_avail":84887660,"statfs":{"total":96636764160,"available":86924963840,"internally_reserved":0,"allocated":7745515520,"data_stored":12683487537,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":983010,"internal_metadata":1965293598},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,6,4,6,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":1.0629999999999999,"15min":1.0029999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":4.2380000000000004,"5min":9.548,"15min":14.773},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.504,"5min":0.70899999999999996,"15min":0.95799999999999996},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.20799999999999999},"max":{"1min":1.847,"5min":9.593,"15min":10.161},"last":0.27400000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.68799999999999994,"15min":0.875},"min":{"1min":0.27400000000000002,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":1.677,"5min":6.7370000000000001,"15min":9.5099999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.056,"5min":0.90300000000000002,"15min":0.78700000000000003},"min":{"1min":0.20200000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":4.3940000000000001,"5min":9.2200000000000006,"15min":9.2200000000000006},"last":0.375}]}]},{"osd":0,"up_from":8,"seq":34359739156,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5809892,"kb_used_data":4613652,"kb_used_omap":592,"kb_used_meta":1195631,"kb_avail":88561948,"statfs":{"total":96636764160,"available":90687434752,"internally_reserved":0,"allocated":4724379648,"data_stored":7733339539,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1224326754},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.72299999999999998},"min":{"1min":0.19900000000000001,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":4.2210000000000001,"5min":6.6829999999999998,"15min":6.7480000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.99199999999999999,"15min":1.0840000000000001},"min":{"1min":0.27000000000000002,"5min":0.191,"15min":0.191},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":9.077},"last":9.4849999999999994}]},{"osd":2,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.218,"15min":1.1759999999999999},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":12.622999999999999,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0269999999999999,"15min":0.98399999999999999},"min":{"1min":0.20999999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":6.319,"5min":8.0009999999999994,"15min":9.1259999999999994},"last":9.5310000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4723507200,"data_stored":7732545395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7744946176,"data_stored":12682988588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3023319040,"data_stored":4952323257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:05:26.245 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-14T01:05:26.628 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:26.627+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:05:26.634 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:26.631+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:27.099 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:05:27.100 INFO:teuthology.orchestra.run.smithi183.stdout:{"epoch":119,"fsid":"01fe18cb-485e-439b-9438-7a52435d33f2","created":"2023-10-13T23:59:26.603673+0000","modified":"2023-10-14T01:05:23.281385+0000","last_up_change":"2023-10-13T23:59:35.733023+0000","last_in_change":"2023-10-13T23:59:27.517541+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-13T23:59:37.286490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-13T23:59:41.813626+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":64,"snap_epoch":96,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c0949fc6-f124-483d-bc0d-21287eb4d027","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6801","nonce":973214594}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6803","nonce":973214594}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6807","nonce":973214594}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":973214594},{"type":"v1","addr":"172.21.15.183:6805","nonce":973214594}]},"public_addr":"172.21.15.183:6801/973214594","cluster_addr":"172.21.15.183:6803/973214594","heartbeat_back_addr":"172.21.15.183:6807/973214594","heartbeat_front_addr":"172.21.15.183:6805/973214594","state":["exists","up"]},{"osd":1,"uuid":"defa036c-6c18-43a4-87d4-a66a3cb829bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6811","nonce":3579441550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6815","nonce":3579441550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6821","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6823","nonce":3579441550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6817","nonce":3579441550},{"type":"v1","addr":"172.21.15.183:6819","nonce":3579441550}]},"public_addr":"172.21.15.183:6811/3579441550","cluster_addr":"172.21.15.183:6815/3579441550","heartbeat_back_addr":"172.21.15.183:6823/3579441550","heartbeat_front_addr":"172.21.15.183:6819/3579441550","state":["exists","up"]},{"osd":2,"uuid":"db6b0fb5-f6a4-4e74-b34a-4cb459386346","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6810","nonce":869421319}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6814","nonce":869421319}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6820","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6822","nonce":869421319}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6816","nonce":869421319},{"type":"v1","addr":"172.21.15.183:6818","nonce":869421319}]},"public_addr":"172.21.15.183:6810/869421319","cluster_addr":"172.21.15.183:6814/869421319","heartbeat_back_addr":"172.21.15.183:6822/869421319","heartbeat_front_addr":"172.21.15.183:6818/869421319","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:32.985796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.024705+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-13T23:59:33.041682+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.183:0/2795815641":"2023-10-14T01:34:04.880809+0000","172.21.15.183:0/3799504759":"2023-10-14T01:11:26.392217+0000","172.21.15.183:0/301131838":"2023-10-14T01:09:59.270351+0000","172.21.15.183:0/4117264617":"2023-10-14T01:35:16.764477+0000","172.21.15.183:0/557665774":"2023-10-14T01:23:07.488857+0000","172.21.15.183:0/1529585824":"2023-10-14T01:13:39.959512+0000","172.21.15.183:0/1692311500":"2023-10-14T01:10:14.461063+0000","172.21.15.183:0/1555743736":"2023-10-14T01:06:33.521007+0000","172.21.15.183:0/4182509406":"2023-10-14T01:12:38.622987+0000","172.21.15.183:0/870020856":"2023-10-14T01:25:32.089259+0000","172.21.15.183:0/2306948515":"2023-10-14T01:07:45.755708+0000","172.21.15.183:0/981386002":"2023-10-14T01:29:17.042976+0000","172.21.15.183:0/399520586":"2023-10-14T01:08:46.819612+0000","172.21.15.183:0/4061210612":"2023-10-14T01:36:44.384618+0000","172.21.15.183:0/3223512488":"2023-10-14T01:30:28.999180+0000","172.21.15.183:0/2810373017":"2023-10-14T01:36:28.846073+0000","172.21.15.183:0/4188019524":"2023-10-14T01:17:16.669593+0000","172.21.15.183:0/2475154413":"2023-10-14T01:14:52.128949+0000","172.21.15.183:0/1180091150":"2023-10-14T01:18:29.289338+0000","172.21.15.183:0/193461531":"2023-10-14T01:20:54.493514+0000","172.21.15.183:0/176881915":"2023-10-14T01:31:41.209835+0000","172.21.15.183:0/323219900":"2023-10-14T01:19:41.436323+0000","172.21.15.183:0/2831995567":"2023-10-14T01:21:55.672335+0000","172.21.15.183:0/1395831165":"2023-10-14T01:16:04.182463+0000","172.21.15.183:0/583246470":"2023-10-14T01:24:20.436738+0000","172.21.15.183:0/4019572365":"2023-10-14T01:26:55.076730+0000","172.21.15.183:0/4068544102":"2023-10-14T01:28:05.833361+0000","172.21.15.183:0/2403304423":"2023-10-14T01:32:53.356723+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-14T01:05:27.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:27.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:27.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:27.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:27.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:27.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:28.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:28.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:28.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:28.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:28.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:28.007+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:28.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:28.007+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:28.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:28.023+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:28.148 INFO:tasks.ceph:Scrubbing osd.0 2023-10-14T01:05:28.149 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-14T01:05:28.522 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-10-14T01:05:28.522 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T01:05:28.523 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-10-14T01:05:28.568 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-14T01:05:29.438 INFO:teuthology.orchestra.run.smithi183.stderr:instructed osd(s) 0 to deep-scrub 2023-10-14T01:05:29.496 INFO:tasks.ceph:Scrubbing osd.1 2023-10-14T01:05:29.496 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-14T01:05:29.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:29.951+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:29.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:29.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:29.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:29.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:30.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:30.011+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:30.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:30.019+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:30.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:30.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:30.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:30.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:30.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:30.047+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:30.059 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-10-14T01:05:30.060 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T01:05:30.060 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-10-14T01:05:30.118 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-14T01:05:30.901 INFO:teuthology.orchestra.run.smithi183.stderr:instructed osd(s) 1 to deep-scrub 2023-10-14T01:05:30.939 INFO:tasks.ceph:Scrubbing osd.2 2023-10-14T01:05:30.940 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-14T01:05:31.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:31.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:31.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:31.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:31.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:31.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.199+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:31.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.199+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:31.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:31.211+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:31.310 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2023-10-14T01:05:31.311 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-14T01:05:31.311 INFO:teuthology.orchestra.run.smithi183.stdout:} 2023-10-14T01:05:31.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-14T01:05:31.750 INFO:teuthology.orchestra.run.smithi183.stderr:instructed osd(s) 2 to deep-scrub 2023-10-14T01:05:31.774 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:05:32.386 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:05:32.387 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:05:32.439 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2094,"stamp":"2023-10-14T01:05:30.543814+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26859028044,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1929562,"num_read_kb":7492555,"num_write":1893756,"num_write_kb":7574813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80542,"ondisk_log_size":80542,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19319904,"kb_used_data":15148160,"kb_used_omap":1902,"kb_used_meta":4169809,"kb_avail":263795616,"statfs":{"total":289910292480,"available":270126710784,"internally_reserved":0,"allocated":15511715840,"data_stored":25387032728,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1947695,"internal_metadata":4269885393},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,2,14,12,15,14],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16097280,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3597,"num_read_kb":14348,"num_write":3912,"num_write_kb":15648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005679"},"pg_stats":[{"pgid":"2.7","version":"119'222378","reported_seq":442496,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:26.714911+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:05:26.714911+0000","last_peered":"2023-10-14T01:05:26.714911+0000","last_clean":"2023-10-14T01:05:26.714911+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:05:26.714911+0000","last_undegraded":"2023-10-14T01:05:26.714911+0000","last_fullsized":"2023-10-14T01:05:26.714911+0000","mapping_epoch":13,"log_start":"116'212311","ondisk_log_start":"116'212311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:25:08.590095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3126980791,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":220995,"num_read_kb":869524,"num_write":221322,"num_write_kb":885273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"119'233153","reported_seq":461371,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:26.702232+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:05:26.702232+0000","last_peered":"2023-10-14T01:05:26.702232+0000","last_clean":"2023-10-14T01:05:26.702232+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:05:26.702232+0000","last_undegraded":"2023-10-14T01:05:26.702232+0000","last_fullsized":"2023-10-14T01:05:26.702232+0000","mapping_epoch":13,"log_start":"116'223108","ondisk_log_start":"116'223108","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:48:19.383773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3150434571,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":230955,"num_read_kb":886813,"num_write":221505,"num_write_kb":885619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"119'238416","reported_seq":473699,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:26.722905+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:05:26.722905+0000","last_peered":"2023-10-14T01:05:26.722905+0000","last_clean":"2023-10-14T01:05:26.722905+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:05:26.722905+0000","last_undegraded":"2023-10-14T01:05:26.722905+0000","last_fullsized":"2023-10-14T01:05:26.722905+0000","mapping_epoch":13,"log_start":"116'228412","ondisk_log_start":"116'228412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T07:59:21.530274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3313402411,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":236379,"num_read_kb":926370,"num_write":233730,"num_write_kb":934799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"119'251949","reported_seq":495899,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:26.678548+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:05:26.678548+0000","last_peered":"2023-10-14T01:05:26.678548+0000","last_clean":"2023-10-14T01:05:26.678548+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:05:26.678548+0000","last_undegraded":"2023-10-14T01:05:26.678548+0000","last_fullsized":"2023-10-14T01:05:26.678548+0000","mapping_epoch":13,"log_start":"116'241910","ondisk_log_start":"116'241910","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:18:04.294509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3404018095,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":245512,"num_read_kb":949495,"num_write":240300,"num_write_kb":960882,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'242893","reported_seq":485835,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:29.020455+0000","last_change":"2023-10-14T00:16:06.071834+0000","last_active":"2023-10-14T01:05:29.020455+0000","last_peered":"2023-10-14T01:05:29.020455+0000","last_clean":"2023-10-14T01:05:29.020455+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:05:29.020455+0000","last_undegraded":"2023-10-14T01:05:29.020455+0000","last_fullsized":"2023-10-14T01:05:29.020455+0000","mapping_epoch":13,"log_start":"116'232812","ondisk_log_start":"116'232812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:53:17.523335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3356549591,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":244373,"num_read_kb":937847,"num_write":237528,"num_write_kb":949605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"119'254362","reported_seq":506563,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:25.495029+0000","last_change":"2023-10-14T00:16:06.066447+0000","last_active":"2023-10-14T01:05:25.495029+0000","last_peered":"2023-10-14T01:05:25.495029+0000","last_clean":"2023-10-14T01:05:25.495029+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:05:25.495029+0000","last_undegraded":"2023-10-14T01:05:25.495029+0000","last_fullsized":"2023-10-14T01:05:25.495029+0000","mapping_epoch":13,"log_start":"116'244311","ondisk_log_start":"116'244311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:15:31.599191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3561492900,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":253376,"num_read_kb":988508,"num_write":249863,"num_write_kb":999278,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"119'251859","reported_seq":500139,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:25.463920+0000","last_change":"2023-10-14T00:16:06.066485+0000","last_active":"2023-10-14T01:05:25.463920+0000","last_peered":"2023-10-14T01:05:25.463920+0000","last_clean":"2023-10-14T01:05:25.463920+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:05:25.463920+0000","last_undegraded":"2023-10-14T01:05:25.463920+0000","last_fullsized":"2023-10-14T01:05:25.463920+0000","mapping_epoch":13,"log_start":"116'241813","ondisk_log_start":"116'241813","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:40:19.002518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3513045345,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":249517,"num_read_kb":973754,"num_write":247354,"num_write_kb":989226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"119'257098","reported_seq":503834,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:26.679772+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:05:26.679772+0000","last_peered":"2023-10-14T01:05:26.679772+0000","last_clean":"2023-10-14T01:05:26.679772+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:05:26.679772+0000","last_undegraded":"2023-10-14T01:05:26.679772+0000","last_fullsized":"2023-10-14T01:05:26.679772+0000","mapping_epoch":13,"log_start":"116'247014","ondisk_log_start":"116'247014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T04:01:06.210461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3432513972,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":248329,"num_read_kb":960135,"num_write":241985,"num_write_kb":967583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"53'125","reported_seq":310,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-10-14T01:05:06.300574+0000","last_change":"2023-10-13T23:59:38.935251+0000","last_active":"2023-10-14T01:05:06.300574+0000","last_peered":"2023-10-14T01:05:06.300574+0000","last_clean":"2023-10-14T01:05:06.300574+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:05:06.300574+0000","last_undegraded":"2023-10-14T01:05:06.300574+0000","last_fullsized":"2023-10-14T01:05:06.300574+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:37.743218+0000","last_clean_scrub_stamp":"2023-10-13T23:59:37.743218+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:21:33.168386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":26858437676,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1929436,"num_read_kb":7492446,"num_write":1893587,"num_write_kb":7572265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15509790720,"data_stored":25385875544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80417,"ondisk_log_size":80417,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739156,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":3945252,"kb_used_data":2956504,"kb_used_omap":349,"kb_used_meta":988386,"kb_avail":90426588,"statfs":{"total":96636764160,"available":92596826112,"internally_reserved":0,"allocated":3027460096,"data_stored":4955852985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1012107601},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.82199999999999995,"15min":0.75900000000000001},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":3.7749999999999999,"5min":16.474,"15min":16.474},"last":0.318},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.70599999999999996,"15min":0.83299999999999996},"min":{"1min":0.21099999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.88700000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.248}]},{"osd":1,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":1.641,"15min":1.845},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":13.519,"5min":13.519,"15min":14.303000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":2.2389999999999999,"5min":1.6870000000000001,"15min":1.905},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":13.565,"5min":13.565,"15min":14.35},"last":0.38500000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739149,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9537312,"kb_used_data":7572692,"kb_used_omap":959,"kb_used_meta":1963648,"kb_avail":84834528,"statfs":{"total":96636764160,"available":86870556672,"internally_reserved":0,"allocated":7754436608,"data_stored":12692400716,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":983010,"internal_metadata":2010775582},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,11,4,10,14],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":1.0629999999999999,"15min":1.0029999999999999},"min":{"1min":0.184,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":4.2380000000000004,"5min":9.548,"15min":14.773},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.70899999999999996,"15min":0.95799999999999996},"min":{"1min":0.24299999999999999,"5min":0.21299999999999999,"15min":0.20799999999999999},"max":{"1min":1.847,"5min":9.593,"15min":10.161},"last":0.39400000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:04:35 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.68799999999999994,"15min":0.875},"min":{"1min":0.27400000000000002,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":1.677,"5min":6.7370000000000001,"15min":9.5099999999999998},"last":0.42299999999999999},{"interface":"front","average":{"1min":1.056,"5min":0.90300000000000002,"15min":0.78700000000000003},"min":{"1min":0.20200000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":4.3940000000000001,"5min":9.2200000000000006,"15min":9.2200000000000006},"last":0.33000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739157,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5837340,"kb_used_data":4618964,"kb_used_omap":592,"kb_used_meta":1217775,"kb_avail":88534500,"statfs":{"total":96636764160,"available":90659328000,"internally_reserved":0,"allocated":4729819136,"data_stored":7738779027,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1247002210},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70199999999999996,"15min":0.72299999999999998},"min":{"1min":0.19900000000000001,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":4.2210000000000001,"5min":6.6829999999999998,"15min":6.7480000000000002},"last":2.2999999999999998},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.99199999999999999,"15min":1.0840000000000001},"min":{"1min":0.27000000000000002,"5min":0.191,"15min":0.191},"max":{"1min":7.6399999999999997,"5min":7.6399999999999997,"15min":9.077},"last":0.38100000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:04:43 2023","interfaces":[{"interface":"back","average":{"1min":1.7150000000000001,"5min":1.218,"15min":1.1759999999999999},"min":{"1min":0.20799999999999999,"5min":0.185,"15min":0.185},"max":{"1min":12.622999999999999,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":1.0269999999999999,"15min":0.98399999999999999},"min":{"1min":0.20999999999999999,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":6.319,"5min":8.0009999999999994,"15min":9.1259999999999994},"last":0.41499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4729004032,"data_stored":7738042227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7753957376,"data_stored":12691999788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3026829312,"data_stored":4955833529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:05:32.443 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.443 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.443 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.444 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.444 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.445 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.445 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.445 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.446 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-13T23:59:37.743218+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=37, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:32.446 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:05:32.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:32.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:32.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:32.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:32.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:32.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:32.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:32.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:32.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:32.751+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:32.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:32.751+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:32.759 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:32.755+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:33.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.095+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:33.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:33.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:33.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:33.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:33.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:33.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.147+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:33.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.155+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:33.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.171+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:33.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.319+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:33.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:33.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:33.379+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:36.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.094+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:05:36.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.118+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:36.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:36.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:36.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:36.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:36.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:36.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.130+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:36.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.130+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:36.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:36.130+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:38.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:38.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:38.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:38.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:38.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:38.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:38.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:38.570+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:38.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:38.570+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:38.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:38.570+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:40.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.686+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:40.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:40.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:40.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.762+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:40.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:40.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.786+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:40.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.786+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:40.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:40.794+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:42.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:41.998+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:05:42.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:41.998+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:05:42.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.014+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:42.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:42.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:42.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:42.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:42.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:42.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.074+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:42.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.074+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:42.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.082+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:42.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.182+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:42.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:42.210+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:44.810 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:44.806+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:45.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:45.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:45.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:45.638+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:49.152 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:49.150+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:05:49.178 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:49.178+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:49.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:49.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:49.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:49.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:49.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.286+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:49.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.286+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:49.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.286+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:49.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:49.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:49.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:49.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:49.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:49.574+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:50.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:50.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:50.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:50.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.518+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:50.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:50.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:50.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:50.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:50.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.894+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:50.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:50.914+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:51.632 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:51.630+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:05:51.660 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:51.658+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:05:51.721 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:51.718+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:51.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:51.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:51.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:51.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:52.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:52.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:52.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:52.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:52.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:52.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:52.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:52.054+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:52.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:52.086+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:52.447 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:05:52.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:52.966+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:53.046 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:05:53.047 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:05:53.106 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2105,"stamp":"2023-10-14T01:05:52.564329+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26900483660,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1939367,"num_read_kb":7531671,"num_write":1903869,"num_write_kb":7615265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80525,"ondisk_log_size":80525,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19685816,"kb_used_data":15225312,"kb_used_omap":1902,"kb_used_meta":4458577,"kb_avail":263429704,"statfs":{"total":289910292480,"available":269752016896,"internally_reserved":0,"allocated":15590719488,"data_stored":25466036376,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1947695,"internal_metadata":4565583825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,13,13,20,20],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24809472,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5831,"num_read_kb":23260,"num_write":6057,"num_write_kb":24228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.016161"},"pg_stats":[{"pgid":"2.7","version":"119'223589","reported_seq":444893,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:52.385051+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:05:52.385051+0000","last_peered":"2023-10-14T01:05:52.385051+0000","last_clean":"2023-10-14T01:05:52.385051+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:05:52.385051+0000","last_undegraded":"2023-10-14T01:05:52.385051+0000","last_fullsized":"2023-10-14T01:05:52.385051+0000","mapping_epoch":13,"log_start":"116'213511","ondisk_log_start":"116'213511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3131953335,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":222179,"num_read_kb":874244,"num_write":222533,"num_write_kb":890117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"119'234468","reported_seq":463884,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:52.388973+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:05:52.388973+0000","last_peered":"2023-10-14T01:05:52.388973+0000","last_clean":"2023-10-14T01:05:52.388973+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:05:52.388973+0000","last_undegraded":"2023-10-14T01:05:52.388973+0000","last_fullsized":"2023-10-14T01:05:52.388973+0000","mapping_epoch":13,"log_start":"116'224408","ondisk_log_start":"116'224408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3155550475,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":232136,"num_read_kb":891529,"num_write":222751,"num_write_kb":890603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"119'239741","reported_seq":476235,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:52.408459+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:05:52.408459+0000","last_peered":"2023-10-14T01:05:52.408459+0000","last_clean":"2023-10-14T01:05:52.408459+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:05:52.408459+0000","last_undegraded":"2023-10-14T01:05:52.408459+0000","last_fullsized":"2023-10-14T01:05:52.408459+0000","mapping_epoch":13,"log_start":"116'229712","ondisk_log_start":"116'229712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3318747691,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":237586,"num_read_kb":931190,"num_write":235032,"num_write_kb":940007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"119'253280","reported_seq":498497,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:52.406419+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:05:52.406419+0000","last_peered":"2023-10-14T01:05:52.406419+0000","last_clean":"2023-10-14T01:05:52.406419+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:05:52.406419+0000","last_undegraded":"2023-10-14T01:05:52.406419+0000","last_fullsized":"2023-10-14T01:05:52.406419+0000","mapping_epoch":13,"log_start":"117'243210","ondisk_log_start":"117'243210","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3409183151,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":246763,"num_read_kb":954487,"num_write":241565,"num_write_kb":965942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'243944","reported_seq":487911,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:05:49.170602+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:05:49.170602+0000","last_peered":"2023-10-14T01:05:49.170602+0000","last_clean":"2023-10-14T01:05:49.170602+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:05:49.170602+0000","last_undegraded":"2023-10-14T01:05:49.170602+0000","last_fullsized":"2023-10-14T01:05:49.170602+0000","mapping_epoch":13,"log_start":"116'233912","ondisk_log_start":"116'233912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":57,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3360616919,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":245372,"num_read_kb":941835,"num_write":238521,"num_write_kb":953577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"119'255765","reported_seq":509332,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:05:50.872598+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:05:50.872598+0000","last_peered":"2023-10-14T01:05:50.872598+0000","last_clean":"2023-10-14T01:05:50.872598+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:05:50.872598+0000","last_undegraded":"2023-10-14T01:05:50.872598+0000","last_fullsized":"2023-10-14T01:05:50.872598+0000","mapping_epoch":13,"log_start":"117'245711","ondisk_log_start":"117'245711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":54,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 18s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3567051172,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":254725,"num_read_kb":993880,"num_write":251226,"num_write_kb":1004730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"119'253255","reported_seq":502840,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:05:50.866406+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:05:50.866406+0000","last_peered":"2023-10-14T01:05:50.866406+0000","last_clean":"2023-10-14T01:05:50.866406+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:05:50.866406+0000","last_undegraded":"2023-10-14T01:05:50.866406+0000","last_fullsized":"2023-10-14T01:05:50.866406+0000","mapping_epoch":13,"log_start":"117'243213","ondisk_log_start":"117'243213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":32,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3518636385,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":250800,"num_read_kb":978870,"num_write":248722,"num_write_kb":994698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"119'258547","reported_seq":506657,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:52.362734+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:05:52.362734+0000","last_peered":"2023-10-14T01:05:52.362734+0000","last_clean":"2023-10-14T01:05:52.362734+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:05:52.362734+0000","last_undegraded":"2023-10-14T01:05:52.362734+0000","last_fullsized":"2023-10-14T01:05:52.362734+0000","mapping_epoch":13,"log_start":"117'248514","ondisk_log_start":"117'248514","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3438154164,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":249680,"num_read_kb":965527,"num_write":243350,"num_write_kb":973043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":321,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:31.470034+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:05:31.470034+0000","last_peered":"2023-10-14T01:05:31.470034+0000","last_clean":"2023-10-14T01:05:31.470034+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:05:31.470034+0000","last_undegraded":"2023-10-14T01:05:31.470034+0000","last_fullsized":"2023-10-14T01:05:31.470034+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":26899893292,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1939241,"num_read_kb":7531562,"num_write":1903700,"num_write_kb":7612717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15589244928,"data_stored":25465329752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80398,"ondisk_log_size":80398,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739161,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4033704,"kb_used_data":2972768,"kb_used_omap":349,"kb_used_meta":1060578,"kb_avail":90338136,"statfs":{"total":96636764160,"available":92506251264,"internally_reserved":0,"allocated":3044114432,"data_stored":4972507321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1086032209},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,4,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.82199999999999995,"15min":0.75900000000000001},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":3.7749999999999999,"5min":16.474,"15min":16.474},"last":0.248},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.70599999999999996,"15min":0.83299999999999996},"min":{"1min":0.21099999999999999,"5min":0.187,"15min":0.187},"max":{"1min":0.88700000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.22900000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:04:55 2023","interfaces":[{"interface":"back","average":{"1min":2.1850000000000001,"5min":1.641,"15min":1.845},"min":{"1min":0.245,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":13.519,"5min":13.519,"15min":14.303000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":2.2389999999999999,"5min":1.6870000000000001,"15min":1.905},"min":{"1min":0.32200000000000001,"5min":0.252,"15min":0.252},"max":{"1min":13.565,"5min":13.565,"15min":14.35},"last":0.317}]}]},{"osd":1,"up_from":8,"seq":34359739154,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9718704,"kb_used_data":7613864,"kb_used_omap":959,"kb_used_meta":2103872,"kb_avail":84653136,"statfs":{"total":96636764160,"available":86684811264,"internally_reserved":0,"allocated":7796596736,"data_stored":12734560844,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":983010,"internal_metadata":2154364958},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,8,7,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:05:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0329999999999999,"15min":0.995},"min":{"1min":0.192,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":8.843,"5min":9.548,"15min":14.773},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.64200000000000002,"15min":0.91300000000000003},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.0880000000000001,"5min":9.593,"15min":10.161},"last":0.627}]},{"osd":2,"last update":"Sat Oct 14 01:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61899999999999999,"15min":0.82899999999999996},"min":{"1min":0.253,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":4.1260000000000003,"5min":6.7370000000000001,"15min":7.3479999999999999},"last":6.431},{"interface":"front","average":{"1min":1.258,"5min":0.99299999999999999,"15min":0.76100000000000001},"min":{"1min":0.20100000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":7.9560000000000004,"5min":9.2200000000000006,"15min":9.2200000000000006},"last":0.247}]}]},{"osd":0,"up_from":8,"seq":34359739161,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5933408,"kb_used_data":4638680,"kb_used_omap":592,"kb_used_meta":1294127,"kb_avail":88438432,"statfs":{"total":96636764160,"available":90560954368,"internally_reserved":0,"allocated":4750008320,"data_stored":7758968211,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1325186658},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,9,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.71699999999999997,"15min":0.72199999999999998},"min":{"1min":0.214,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":2.2999999999999998,"5min":6.6829999999999998,"15min":6.7480000000000002},"last":0.318},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1419999999999999,"15min":1.143},"min":{"1min":0.25600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":9.4849999999999994,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.3120000000000001,"15min":1.1559999999999999},"min":{"1min":0.23000000000000001,"5min":0.185,"15min":0.185},"max":{"1min":6.5030000000000001,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.27900000000000003},{"interface":"front","average":{"1min":1.419,"5min":1.173,"15min":1.048},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":9.5310000000000006,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.26100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4749094912,"data_stored":7758133107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7796649984,"data_stored":12734692396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3043500032,"data_stored":4972504249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:05:53.107 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.108 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.108 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.108 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.109 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.109 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.109 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.110 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:05:53.110 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:05:54.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.182+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:54.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:54.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:54.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:54.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:54.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:54.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.246+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:54.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:54.266+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:55.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:55.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:55.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:55.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:55.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:55.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.306+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:55.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.306+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:55.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.314+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:55.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.682+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:05:55.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.682+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:05:55.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.702+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:55.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:55.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:55.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:55.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:55.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:55.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.722+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:55.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.734+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:55.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:55.738+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:56.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:56.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:56.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:56.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:56.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:56.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.310+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:56.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.310+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:56.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.330+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:56.366 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:56.362+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:05:56.402 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:56.398+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:56.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:56.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:56.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:56.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:56.574+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:57.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:57.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:57.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:57.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.190+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:57.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.190+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:57.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.194+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:57.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.326+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:05:57.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:57.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:57.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:05:57.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:05:57.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.422+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:05:57.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.422+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:05:57.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.422+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:05:57.688 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:05:57.686+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:05:57.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:05:57.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:05:57.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:05:57.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:05:57.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:00.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:00.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:00.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:00.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:00.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:00.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:00.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:00.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:00.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:00.230+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:00.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:00.230+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:00.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:00.250+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:01.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:01.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:01.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:01.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:01.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:01.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.562+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:01.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.562+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:01.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:01.622+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:04.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.526+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:04.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:04.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:04.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:04.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:04.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:04.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.574+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:04.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.574+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:04.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:04.582+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:05.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.342+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:05.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:05.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:05.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:05.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:05.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:05.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.442+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:05.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.442+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:05.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:05.442+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:07.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:07.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:07.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:07.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.978+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:07.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:07.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.990+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:07.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.990+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:07.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:07.990+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:08.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.086+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:08.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:08.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:08.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:08.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.170+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:08.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.202+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:08.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.210+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:06:08.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.210+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:06:08.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.222+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:08.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:08.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:08.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:08.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:08.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.266+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:08.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:08.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:08.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:08.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:08.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:08.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.602+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:08.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:08.618+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:09.995 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:09.994+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:06:09.997 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:09.994+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:06:10.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:10.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:10.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:10.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:10.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:10.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.070+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:10.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.070+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:10.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:10.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:10.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:10.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.450+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:10.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.910+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:10.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:10.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:10.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:10.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:10.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:10.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:10.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:10.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:10.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:11.020 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:11.018+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:06:11.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:11.974+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:11.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:11.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:12.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:12.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:12.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:12.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:12.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.018+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:12.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.018+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:12.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:12.024 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:12.022+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:06:12.032 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:12.030+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:06:12.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.222+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:12.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:12.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:12.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:12.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:12.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.310+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:12.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.310+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:12.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:12.310+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:13.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:06:13.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.450+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:13.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:13.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:13.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:13.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:13.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.494+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:13.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.494+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:13.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:13.502+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:13.872 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:06:13.873 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:06:13.907 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2115,"stamp":"2023-10-14T01:06:12.567972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26928971340,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1946137,"num_read_kb":7558663,"num_write":1910791,"num_write_kb":7642953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80555,"ondisk_log_size":80555,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19965664,"kb_used_data":15283016,"kb_used_omap":1903,"kb_used_meta":4680720,"kb_avail":263149856,"statfs":{"total":289910292480,"available":269465452544,"internally_reserved":0,"allocated":15649808384,"data_stored":25525125272,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":4793057602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,1,5,4,15,18,10,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19333120,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4520,"num_read_kb":18016,"num_write":4694,"num_write_kb":18776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002358"},"pg_stats":[{"pgid":"2.7","version":"119'224299","reported_seq":446305,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:07.737059+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:06:07.737059+0000","last_peered":"2023-10-14T01:06:07.737059+0000","last_clean":"2023-10-14T01:06:07.737059+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:06:07.737059+0000","last_undegraded":"2023-10-14T01:06:07.737059+0000","last_fullsized":"2023-10-14T01:06:07.737059+0000","mapping_epoch":13,"log_start":"116'214211","ondisk_log_start":"116'214211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3134857399,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":222879,"num_read_kb":877040,"num_write":223243,"num_write_kb":892957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"119'235238","reported_seq":465377,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:07.742177+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:06:07.742177+0000","last_peered":"2023-10-14T01:06:07.742177+0000","last_clean":"2023-10-14T01:06:07.742177+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:06:07.742177+0000","last_undegraded":"2023-10-14T01:06:07.742177+0000","last_fullsized":"2023-10-14T01:06:07.742177+0000","mapping_epoch":13,"log_start":"117'225208","ondisk_log_start":"117'225208","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3158548747,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":232850,"num_read_kb":894373,"num_write":223479,"num_write_kb":893515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"119'240525","reported_seq":477767,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:07.753199+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:06:07.753199+0000","last_peered":"2023-10-14T01:06:07.753199+0000","last_clean":"2023-10-14T01:06:07.753199+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:06:07.753199+0000","last_undegraded":"2023-10-14T01:06:07.753199+0000","last_fullsized":"2023-10-14T01:06:07.753199+0000","mapping_epoch":13,"log_start":"117'230512","ondisk_log_start":"117'230512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3321905707,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":238328,"num_read_kb":934150,"num_write":235802,"num_write_kb":943087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"119'254099","reported_seq":500128,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:07.682874+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:06:07.682874+0000","last_peered":"2023-10-14T01:06:07.682874+0000","last_clean":"2023-10-14T01:06:07.682874+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:06:07.682874+0000","last_undegraded":"2023-10-14T01:06:07.682874+0000","last_fullsized":"2023-10-14T01:06:07.682874+0000","mapping_epoch":13,"log_start":"117'244010","ondisk_log_start":"117'244010","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3412361647,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":247567,"num_read_kb":957691,"num_write":242342,"num_write_kb":969050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'244991","reported_seq":490001,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:09.316522+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:06:09.316522+0000","last_peered":"2023-10-14T01:06:09.316522+0000","last_clean":"2023-10-14T01:06:09.316522+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:06:09.316522+0000","last_undegraded":"2023-10-14T01:06:09.316522+0000","last_fullsized":"2023-10-14T01:06:09.316522+0000","mapping_epoch":13,"log_start":"117'234912","ondisk_log_start":"117'234912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":135,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 38s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3364721111,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":246385,"num_read_kb":945871,"num_write":239506,"num_write_kb":957517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"119'256918","reported_seq":511537,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:11.301671+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:06:11.301671+0000","last_peered":"2023-10-14T01:06:11.301671+0000","last_clean":"2023-10-14T01:06:11.301671+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:06:11.301671+0000","last_undegraded":"2023-10-14T01:06:11.301671+0000","last_fullsized":"2023-10-14T01:06:11.301671+0000","mapping_epoch":13,"log_start":"117'246911","ondisk_log_start":"117'246911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":106,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 39s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3571593636,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":255757,"num_read_kb":997992,"num_write":252328,"num_write_kb":1009138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"119'254363","reported_seq":504988,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:11.293386+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:06:11.293386+0000","last_peered":"2023-10-14T01:06:11.293386+0000","last_clean":"2023-10-14T01:06:11.293386+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:06:11.293386+0000","last_undegraded":"2023-10-14T01:06:11.293386+0000","last_fullsized":"2023-10-14T01:06:11.293386+0000","mapping_epoch":13,"log_start":"117'244313","ondisk_log_start":"117'244313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":67,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3523002721,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":251823,"num_read_kb":982958,"num_write":249789,"num_write_kb":998966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"119'259386","reported_seq":508250,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:07.693961+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:06:07.693961+0000","last_peered":"2023-10-14T01:06:07.693961+0000","last_clean":"2023-10-14T01:06:07.693961+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:06:07.693961+0000","last_undegraded":"2023-10-14T01:06:07.693961+0000","last_fullsized":"2023-10-14T01:06:07.693961+0000","mapping_epoch":13,"log_start":"117'249314","ondisk_log_start":"117'249314","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3441390004,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":250422,"num_read_kb":968479,"num_write":244133,"num_write_kb":976175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":321,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:31.470034+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:05:31.470034+0000","last_peered":"2023-10-14T01:05:31.470034+0000","last_clean":"2023-10-14T01:05:31.470034+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:05:31.470034+0000","last_undegraded":"2023-10-14T01:05:31.470034+0000","last_fullsized":"2023-10-14T01:05:31.470034+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":26928380972,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1946011,"num_read_kb":7558554,"num_write":1910622,"num_write_kb":7640405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15647584256,"data_stored":25523669080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80428,"ondisk_log_size":80428,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739165,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4112956,"kb_used_data":2985396,"kb_used_omap":349,"kb_used_meta":1127202,"kb_avail":90258884,"statfs":{"total":96636764160,"available":92425097216,"internally_reserved":0,"allocated":3057045504,"data_stored":4985438393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1154255185},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:05:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1,"15min":0.81899999999999995},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":6.0880000000000001,"5min":16.474,"15min":16.474},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.84399999999999997},"min":{"1min":0.20599999999999999,"5min":0.20399999999999999,"15min":0.187},"max":{"1min":5.7050000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.372}]},{"osd":1,"last update":"Sat Oct 14 01:05:58 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.4139999999999999,"15min":1.8129999999999999},"min":{"1min":0.26700000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":8.3849999999999998,"5min":13.519,"15min":14.303000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.46,"15min":1.8720000000000001},"min":{"1min":0.317,"5min":0.252,"15min":0.252},"max":{"1min":8.4800000000000004,"5min":13.565,"15min":14.35},"last":0.63700000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739157,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9809568,"kb_used_data":7639448,"kb_used_omap":961,"kb_used_meta":2169150,"kb_avail":84562272,"statfs":{"total":96636764160,"available":86591766528,"internally_reserved":0,"allocated":7822794752,"data_stored":12760758860,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":2221209999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,1,4,0,8,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:05:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0329999999999999,"15min":0.995},"min":{"1min":0.192,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":8.843,"5min":9.548,"15min":14.773},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.64200000000000002,"15min":0.91300000000000003},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.0880000000000001,"5min":9.593,"15min":10.161},"last":0.24399999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61899999999999999,"15min":0.82899999999999996},"min":{"1min":0.253,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":4.1260000000000003,"5min":6.7370000000000001,"15min":7.3479999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.258,"5min":0.99299999999999999,"15min":0.76100000000000001},"min":{"1min":0.20100000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":7.9560000000000004,"5min":9.2200000000000006,"15min":9.2200000000000006},"last":0.32700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739165,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6043140,"kb_used_data":4658172,"kb_used_omap":592,"kb_used_meta":1384367,"kb_avail":88328700,"statfs":{"total":96636764160,"available":90448588800,"internally_reserved":0,"allocated":4769968128,"data_stored":7778928019,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1417592418},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,3,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.71699999999999997,"15min":0.72199999999999998},"min":{"1min":0.214,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":2.2999999999999998,"5min":6.6829999999999998,"15min":6.7480000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1419999999999999,"15min":1.143},"min":{"1min":0.25600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":9.4849999999999994,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.28000000000000003}]},{"osd":2,"last update":"Sat Oct 14 01:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.3120000000000001,"15min":1.1559999999999999},"min":{"1min":0.23000000000000001,"5min":0.185,"15min":0.185},"max":{"1min":6.5030000000000001,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.23000000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.173,"15min":1.048},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":9.5310000000000006,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.40100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4769075200,"data_stored":7778113395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7821856768,"data_stored":12759899180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3056652288,"data_stored":4985656505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:06:13.909 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.909 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.910 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.910 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.911 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.911 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.911 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.912 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:13.912 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:06:14.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:14.094+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:15.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:15.594+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:15.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:15.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:15.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:15.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:15.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:15.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:15.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:15.606+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:16.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.206+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:16.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:16.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:16.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.298+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:16.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.298+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:16.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:16.298+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:19.010 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:19.009+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:06:19.035 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:19.033+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:06:21.592 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:21.589+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:06:24.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.481+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:24.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:24.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:24.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:24.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.541+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:24.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:24.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:24.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:24.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:24.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.829+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:24.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.829+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:24.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.829+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:24.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:24.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:24.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:24.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:24.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.953+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:24.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:24.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:24.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:25.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:25.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:25.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:25.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.521+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:25.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.521+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:25.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:25.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:25.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:25.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:25.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:25.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:25.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:25.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:25.793+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:26.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:26.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:26.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:26.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:26.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:26.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:26.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:26.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:26.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:26.962 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:26.957+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:06:28.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:28.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:28.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:28.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:28.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:28.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:28.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:28.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:29.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:28.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:29.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:28.993+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:29.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:28.993+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:29.024 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:29.013+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:29.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:29.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:29.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:29.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:29.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:29.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:31.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:31.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:31.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.837+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:31.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:31.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:32.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:32.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:32.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:31.997+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:32.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.001+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:32.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.773+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:32.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:32.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:32.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:32.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:32.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:32.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.813+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:32.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.813+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:32.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:32.829+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:33.913 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:06:34.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:34.281+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:34.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:34.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:34.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:34.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:34.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:34.397+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:34.555 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:06:34.555 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:06:34.646 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2125,"stamp":"2023-10-14T01:06:32.571373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26961628748,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1953846,"num_read_kb":7589395,"num_write":1918764,"num_write_kb":7674845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80594,"ondisk_log_size":80594,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":18933312,"kb_used_data":15346916,"kb_used_omap":1903,"kb_used_meta":3584464,"kb_avail":264182208,"statfs":{"total":289910292480,"available":270522580992,"internally_reserved":0,"allocated":15715241984,"data_stored":25590558872,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":3670491458},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,4,11,11,15,17,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18415616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4384,"num_read_kb":17464,"num_write":4494,"num_write_kb":17976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002091"},"pg_stats":[{"pgid":"2.7","version":"119'225276","reported_seq":448200,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:28.124974+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:06:28.124974+0000","last_peered":"2023-10-14T01:06:28.124974+0000","last_clean":"2023-10-14T01:06:28.124974+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:06:28.124974+0000","last_undegraded":"2023-10-14T01:06:28.124974+0000","last_fullsized":"2023-10-14T01:06:28.124974+0000","mapping_epoch":13,"log_start":"117'215211","ondisk_log_start":"117'215211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3138842807,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":223795,"num_read_kb":880684,"num_write":224220,"num_write_kb":896865,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"119'236294","reported_seq":467381,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:28.116699+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:06:28.116699+0000","last_peered":"2023-10-14T01:06:28.116699+0000","last_clean":"2023-10-14T01:06:28.116699+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:06:28.116699+0000","last_undegraded":"2023-10-14T01:06:28.116699+0000","last_fullsized":"2023-10-14T01:06:28.116699+0000","mapping_epoch":13,"log_start":"117'226208","ondisk_log_start":"117'226208","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3162640651,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":233786,"num_read_kb":898105,"num_write":224478,"num_write_kb":897511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"119'241507","reported_seq":479719,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:28.113082+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:06:28.113082+0000","last_peered":"2023-10-14T01:06:28.113082+0000","last_clean":"2023-10-14T01:06:28.113082+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:06:28.113082+0000","last_undegraded":"2023-10-14T01:06:28.113082+0000","last_fullsized":"2023-10-14T01:06:28.113082+0000","mapping_epoch":13,"log_start":"117'231412","ondisk_log_start":"117'231412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3325854251,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":239295,"num_read_kb":937998,"num_write":236764,"num_write_kb":946935,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"119'255180","reported_seq":502234,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:28.096981+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:06:28.096981+0000","last_peered":"2023-10-14T01:06:28.096981+0000","last_clean":"2023-10-14T01:06:28.096981+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:06:28.096981+0000","last_undegraded":"2023-10-14T01:06:28.096981+0000","last_fullsized":"2023-10-14T01:06:28.096981+0000","mapping_epoch":13,"log_start":"117'245110","ondisk_log_start":"117'245110","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3416547759,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":248577,"num_read_kb":961723,"num_write":243366,"num_write_kb":973146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"119'246061","reported_seq":492026,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:29.414158+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:06:29.414158+0000","last_peered":"2023-10-14T01:06:29.414158+0000","last_clean":"2023-10-14T01:06:29.414158+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:06:29.414158+0000","last_undegraded":"2023-10-14T01:06:29.414158+0000","last_fullsized":"2023-10-14T01:06:29.414158+0000","mapping_epoch":13,"log_start":"117'236012","ondisk_log_start":"117'236012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":177,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 58s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3368919511,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":247316,"num_read_kb":949579,"num_write":240532,"num_write_kb":961621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"119'257960","reported_seq":513532,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:31.513585+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:06:31.513585+0000","last_peered":"2023-10-14T01:06:31.513585+0000","last_clean":"2023-10-14T01:06:31.513585+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:06:31.513585+0000","last_undegraded":"2023-10-14T01:06:31.513585+0000","last_fullsized":"2023-10-14T01:06:31.513585+0000","mapping_epoch":13,"log_start":"117'247911","ondisk_log_start":"117'247911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":170,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 59s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3575644580,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":256695,"num_read_kb":1001740,"num_write":253316,"num_write_kb":1013090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"119'255440","reported_seq":507109,"reported_epoch":119,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:31.510342+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:06:31.510342+0000","last_peered":"2023-10-14T01:06:31.510342+0000","last_clean":"2023-10-14T01:06:31.510342+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:06:31.510342+0000","last_undegraded":"2023-10-14T01:06:31.510342+0000","last_fullsized":"2023-10-14T01:06:31.510342+0000","mapping_epoch":13,"log_start":"117'245413","ondisk_log_start":"117'245413","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":140,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 57s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3527180641,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":252850,"num_read_kb":987050,"num_write":250806,"num_write_kb":1003034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"119'260440","reported_seq":510302,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:06:28.079433+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:06:28.079433+0000","last_peered":"2023-10-14T01:06:28.079433+0000","last_clean":"2023-10-14T01:06:28.079433+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:06:28.079433+0000","last_undegraded":"2023-10-14T01:06:28.079433+0000","last_fullsized":"2023-10-14T01:06:28.079433+0000","mapping_epoch":13,"log_start":"117'250414","ondisk_log_start":"117'250414","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3445408180,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":251406,"num_read_kb":972407,"num_write":245113,"num_write_kb":980095,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":321,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:31.470034+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:05:31.470034+0000","last_peered":"2023-10-14T01:05:31.470034+0000","last_clean":"2023-10-14T01:05:31.470034+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:05:31.470034+0000","last_undegraded":"2023-10-14T01:05:31.470034+0000","last_fullsized":"2023-10-14T01:05:31.470034+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":26961038380,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1953720,"num_read_kb":7589286,"num_write":1918595,"num_write_kb":7672297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15713021952,"data_stored":25589106776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80467,"ondisk_log_size":80467,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739169,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4156932,"kb_used_data":2997436,"kb_used_omap":349,"kb_used_meta":1159138,"kb_avail":90214908,"statfs":{"total":96636764160,"available":92380065792,"internally_reserved":0,"allocated":3069374464,"data_stored":4997767353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1186957649},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,3,4,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:05:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1,"15min":0.81899999999999995},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":6.0880000000000001,"5min":16.474,"15min":16.474},"last":0.219},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.84399999999999997},"min":{"1min":0.20599999999999999,"5min":0.20399999999999999,"15min":0.187},"max":{"1min":5.7050000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.249}]},{"osd":1,"last update":"Sat Oct 14 01:05:58 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.4139999999999999,"15min":1.8129999999999999},"min":{"1min":0.26700000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":8.3849999999999998,"5min":13.519,"15min":14.303000000000001},"last":3.5529999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.46,"15min":1.8720000000000001},"min":{"1min":0.317,"5min":0.252,"15min":0.252},"max":{"1min":8.4800000000000004,"5min":13.565,"15min":14.35},"last":3.589}]}]},{"osd":1,"up_from":8,"seq":34359739161,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8653776,"kb_used_data":7671496,"kb_used_omap":961,"kb_used_meta":981310,"kb_avail":85718064,"statfs":{"total":96636764160,"available":87775297536,"internally_reserved":0,"allocated":7855611904,"data_stored":12793576012,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1004861839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,1,4,6,11,12,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:05:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0329999999999999,"15min":0.995},"min":{"1min":0.192,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":8.843,"5min":9.548,"15min":14.773},"last":0.44},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.64200000000000002,"15min":0.91300000000000003},"min":{"1min":0.222,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.0880000000000001,"5min":9.593,"15min":10.161},"last":0.254}]},{"osd":2,"last update":"Sat Oct 14 01:05:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61899999999999999,"15min":0.82899999999999996},"min":{"1min":0.253,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":4.1260000000000003,"5min":6.7370000000000001,"15min":7.3479999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":1.258,"5min":0.99299999999999999,"15min":0.76100000000000001},"min":{"1min":0.20100000000000001,"5min":0.20000000000000001,"15min":0.193},"max":{"1min":7.9560000000000004,"5min":9.2200000000000006,"15min":9.2200000000000006},"last":0.28699999999999998}]}]},{"osd":0,"up_from":8,"seq":34359739169,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6122604,"kb_used_data":4677984,"kb_used_omap":592,"kb_used_meta":1444015,"kb_avail":88249236,"statfs":{"total":96636764160,"available":90367217664,"internally_reserved":0,"allocated":4790255616,"data_stored":7799215507,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1478671970},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:05:47 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.71699999999999997,"15min":0.72199999999999998},"min":{"1min":0.214,"5min":0.192,"15min":0.16900000000000001},"max":{"1min":2.2999999999999998,"5min":6.6829999999999998,"15min":6.7480000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1419999999999999,"15min":1.143},"min":{"1min":0.25600000000000001,"5min":0.191,"15min":0.191},"max":{"1min":9.4849999999999994,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.29099999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:05:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0289999999999999,"5min":1.3120000000000001,"15min":1.1559999999999999},"min":{"1min":0.23000000000000001,"5min":0.185,"15min":0.185},"max":{"1min":6.5030000000000001,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":1.419,"5min":1.173,"15min":1.048},"min":{"1min":0.254,"5min":0.20799999999999999,"15min":0.182},"max":{"1min":9.5310000000000006,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.33400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4789288960,"data_stored":7798327155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7854878720,"data_stored":12792921132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3068854272,"data_stored":4997858489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:06:34.648 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.648 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.648 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.649 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.649 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.650 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.650 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.651 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:34.651 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:06:38.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.421+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:06:38.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.421+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:06:38.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.433+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:38.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:38.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:38.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:38.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:38.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:38.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.469+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:38.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.469+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:38.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.485+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:38.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.725+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:38.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:38.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:38.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:38.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:38.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:38.737+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:39.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.225+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:39.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:39.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:39.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:39.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:39.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:39.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.269+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:39.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.269+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:39.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.269+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:39.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.773+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:06:39.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.797+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:06:39.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.797+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:39.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:39.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:39.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:39.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:39.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:39.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:39.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:39.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.825+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:39.865 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:39.861+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:06:39.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:39.869+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:06:39.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:39.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:39.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.921+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:39.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:39.997+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:40.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.633+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:40.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:40.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:40.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:40.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:40.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.673+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:40.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.673+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:40.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:40.673+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:44.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.049+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:44.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:44.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:44.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:44.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.085+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:44.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.085+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:44.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.109+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:44.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:44.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:44.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:44.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:44.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:44.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:44.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:44.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:44.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:45.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:45.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:45.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.005+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:45.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.005+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:45.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.629+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:45.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:45.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:45.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:45.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:45.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:45.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.693+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:45.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.693+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:45.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.693+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:45.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:45.957+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:46.270 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:46.269+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:06:46.289 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:06:46.285+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:06:49.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.817+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:49.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:49.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:06:49.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:49.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:49.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:49.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.853+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:49.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.853+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:49.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:49.873+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:51.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.005+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:51.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.005+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:51.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:51.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:51.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:51.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.041+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:51.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.041+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:51.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:51.041+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:52.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.392+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:52.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:52.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:52.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:52.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:52.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:52.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:52.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:52.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:06:53.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:53.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:53.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:53.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:54.652 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:06:55.497 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:06:55.498 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:06:55.524 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2137,"stamp":"2023-10-14T01:06:54.579966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27001044556,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1963448,"num_read_kb":7627703,"num_write":1928410,"num_write_kb":7713429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80547,"ondisk_log_size":80547,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19018968,"kb_used_data":15423744,"kb_used_omap":1903,"kb_used_meta":3593296,"kb_avail":264096552,"statfs":{"total":289910292480,"available":270434869248,"internally_reserved":0,"allocated":15793913856,"data_stored":25669219176,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":3679535426},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,6,1,5,10,13,17,12],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22704128,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5577,"num_read_kb":22236,"num_write":5573,"num_write_kb":22292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007008"},"pg_stats":[{"pgid":"2.7","version":"120'226487","reported_seq":450596,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:06:53.704899+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:06:53.704899+0000","last_peered":"2023-10-14T01:06:53.704899+0000","last_clean":"2023-10-14T01:06:53.704899+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:06:53.704899+0000","last_undegraded":"2023-10-14T01:06:53.704899+0000","last_fullsized":"2023-10-14T01:06:53.704899+0000","mapping_epoch":13,"log_start":"117'216411","ondisk_log_start":"117'216411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3143811255,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":224982,"num_read_kb":885412,"num_write":225431,"num_write_kb":901709,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"120'237548","reported_seq":469822,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:06:53.708967+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:06:53.708967+0000","last_peered":"2023-10-14T01:06:53.708967+0000","last_clean":"2023-10-14T01:06:53.708967+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:06:53.708967+0000","last_undegraded":"2023-10-14T01:06:53.708967+0000","last_fullsized":"2023-10-14T01:06:53.708967+0000","mapping_epoch":13,"log_start":"117'227508","ondisk_log_start":"117'227508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3167478027,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":234960,"num_read_kb":902785,"num_write":225663,"num_write_kb":902251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"120'242749","reported_seq":482213,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:06:53.685671+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:06:53.685671+0000","last_peered":"2023-10-14T01:06:53.685671+0000","last_clean":"2023-10-14T01:06:53.685671+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:06:53.685671+0000","last_undegraded":"2023-10-14T01:06:53.685671+0000","last_fullsized":"2023-10-14T01:06:53.685671+0000","mapping_epoch":13,"log_start":"117'232712","ondisk_log_start":"117'232712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3330834987,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":240538,"num_read_kb":942958,"num_write":237983,"num_write_kb":951811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"120'256590","reported_seq":504912,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:06:53.716623+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:06:53.716623+0000","last_peered":"2023-10-14T01:06:53.716623+0000","last_clean":"2023-10-14T01:06:53.716623+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:06:53.716623+0000","last_undegraded":"2023-10-14T01:06:53.716623+0000","last_fullsized":"2023-10-14T01:06:53.716623+0000","mapping_epoch":13,"log_start":"117'246510","ondisk_log_start":"117'246510","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3422032303,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":249829,"num_read_kb":966723,"num_write":244710,"num_write_kb":978522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"120'247380","reported_seq":494649,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:54.547721+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:06:54.547721+0000","last_peered":"2023-10-14T01:06:54.547721+0000","last_clean":"2023-10-14T01:06:54.547721+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:06:54.547721+0000","last_undegraded":"2023-10-14T01:06:54.547721+0000","last_fullsized":"2023-10-14T01:06:54.547721+0000","mapping_epoch":13,"log_start":"117'237312","ondisk_log_start":"117'237312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":254,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 83s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3373978071,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":248580,"num_read_kb":954619,"num_write":241774,"num_write_kb":966589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"120'259080","reported_seq":515725,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:51.779377+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:06:51.779377+0000","last_peered":"2023-10-14T01:06:51.779377+0000","last_clean":"2023-10-14T01:06:51.779377+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:06:51.779377+0000","last_undegraded":"2023-10-14T01:06:51.779377+0000","last_fullsized":"2023-10-14T01:06:51.779377+0000","mapping_epoch":13,"log_start":"117'249011","ondisk_log_start":"117'249011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 79s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3580068260,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":257750,"num_read_kb":1005956,"num_write":254403,"num_write_kb":1017438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"120'256556","reported_seq":509328,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:06:51.752344+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:06:51.752344+0000","last_peered":"2023-10-14T01:06:51.752344+0000","last_clean":"2023-10-14T01:06:51.752344+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:06:51.752344+0000","last_undegraded":"2023-10-14T01:06:51.752344+0000","last_fullsized":"2023-10-14T01:06:51.752344+0000","mapping_epoch":13,"log_start":"117'246513","ondisk_log_start":"117'246513","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":202,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 77s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3531563361,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":253938,"num_read_kb":991394,"num_write":251875,"num_write_kb":1007310,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"120'261821","reported_seq":513043,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:06:53.677207+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:06:53.677207+0000","last_peered":"2023-10-14T01:06:53.677207+0000","last_clean":"2023-10-14T01:06:53.677207+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:06:53.677207+0000","last_undegraded":"2023-10-14T01:06:53.677207+0000","last_fullsized":"2023-10-14T01:06:53.677207+0000","mapping_epoch":13,"log_start":"117'251814","ondisk_log_start":"117'251814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3450687924,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":252745,"num_read_kb":977747,"num_write":246402,"num_write_kb":985251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":322,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-14T01:05:31.470112+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:05:31.470112+0000","last_peered":"2023-10-14T01:05:31.470112+0000","last_clean":"2023-10-14T01:05:31.470112+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:05:31.470112+0000","last_undegraded":"2023-10-14T01:05:31.470112+0000","last_fullsized":"2023-10-14T01:05:31.470112+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27000454188,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1963322,"num_read_kb":7627594,"num_write":1928241,"num_write_kb":7710881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15792656384,"data_stored":25668741208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80420,"ondisk_log_size":80420,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739173,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4205840,"kb_used_data":3010248,"kb_used_omap":349,"kb_used_meta":1195234,"kb_avail":90166000,"statfs":{"total":96636764160,"available":92329984000,"internally_reserved":0,"allocated":3082493952,"data_stored":5010882985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1223919953},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,1,1,4,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:05:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1,"15min":0.81899999999999995},"min":{"1min":0.20599999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":6.0880000000000001,"5min":16.474,"15min":16.474},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.79100000000000004,"15min":0.84399999999999997},"min":{"1min":0.20599999999999999,"5min":0.20399999999999999,"15min":0.187},"max":{"1min":5.7050000000000001,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.254}]},{"osd":1,"last update":"Sat Oct 14 01:05:58 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.4139999999999999,"15min":1.8129999999999999},"min":{"1min":0.26700000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":8.3849999999999998,"5min":13.519,"15min":14.303000000000001},"last":2.8959999999999999},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.46,"15min":1.8720000000000001},"min":{"1min":0.317,"5min":0.252,"15min":0.252},"max":{"1min":8.4800000000000004,"5min":13.565,"15min":14.35},"last":2.927}]}]},{"osd":1,"up_from":8,"seq":34359739166,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8571788,"kb_used_data":7711108,"kb_used_omap":961,"kb_used_meta":859710,"kb_avail":85800052,"statfs":{"total":96636764160,"available":87859253248,"internally_reserved":0,"allocated":7896174592,"data_stored":12834134844,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":880343439},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,0,5,5,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0109999999999999,"15min":1.0169999999999999},"min":{"1min":0.24399999999999999,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":8.1500000000000004,"5min":9.548,"15min":14.773},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63300000000000001,"15min":0.85999999999999999},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":3.5270000000000001,"5min":9.593,"15min":10.161},"last":0.44400000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:06:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":0.77600000000000002,"15min":0.86099999999999999},"min":{"1min":0.27200000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":10.403,"5min":10.403,"15min":10.403},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.91500000000000004,"15min":0.76100000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":4.1299999999999999,"5min":8.9179999999999993,"15min":9.2200000000000006},"last":0.48799999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739174,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6241340,"kb_used_data":4702388,"kb_used_omap":592,"kb_used_meta":1538351,"kb_avail":88130500,"statfs":{"total":96636764160,"available":90245632000,"internally_reserved":0,"allocated":4815245312,"data_stored":7824201347,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1575272034},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,4,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.66500000000000004,"15min":0.72799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":6.7480000000000002},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.99399999999999999,"15min":1.1439999999999999},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":3.6429999999999998,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.27600000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":1.2470000000000001,"15min":1.087},"min":{"1min":0.218,"5min":0.185,"15min":0.185},"max":{"1min":3.6989999999999998,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.0529999999999999,"15min":1.05},"min":{"1min":0.223,"5min":0.20999999999999999,"15min":0.182},"max":{"1min":3.681,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.248}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4814319616,"data_stored":7823357811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7896469504,"data_stored":12834511916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3081867264,"data_stored":5010871481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:06:55.525 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.526 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.526 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.526 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.527 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.527 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.527 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.528 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:06:55.528 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:06:59.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.196+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:06:59.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:06:59.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:06:59.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:06:59.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:06:59.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:06:59.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.216+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:06:59.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:06:59.220+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:01.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:01.860+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:01.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:01.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:01.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:01.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:01.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:01.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:01.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:01.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:01.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:01.888+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:03.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:03.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:03.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:03.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:03.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.360+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:03.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.360+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:03.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.388+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:03.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.752+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:07:03.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.764+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:07:03.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.780+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:03.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:03.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:03.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:03.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:03.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:03.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.820+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:03.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.820+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:03.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:03.824+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:04.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:04.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:04.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:04.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:04.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:04.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:04.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:04.712+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:05.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:05.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:05.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:05.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:05.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:05.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.888+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:05.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.888+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:05.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:05.916+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:07.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:07.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:07.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:07.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:07.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:07.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.792+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:07.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.792+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:07.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:07.792+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:08.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:08.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:08.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:08.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:08.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:08.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:08.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.268+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:08.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.268+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:08.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:08.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:08.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:08.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:08.588+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:09.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:09.568+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:09.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:09.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:09.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:09.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:09.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:09.632+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:11.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.464+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:11.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:11.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:11.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:11.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:11.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:11.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.528+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:11.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.528+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:11.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:11.532+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:12.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.740+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:12.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:12.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:12.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:12.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:12.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:12.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.768+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:12.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.768+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:12.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:12.768+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:14.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:14.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:14.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:14.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.596+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:14.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.784+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:14.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:14.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:14.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:14.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:14.824+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:15.530 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:07:15.593 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:15.588+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:07:15.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:15.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:15.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:15.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:15.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:15.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.972+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:15.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.972+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:15.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:15.972+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:16.295 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:07:16.296 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:07:16.336 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2147,"stamp":"2023-10-14T01:07:14.583234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27033267788,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1971323,"num_read_kb":7659095,"num_write":1936274,"num_write_kb":7744885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80528,"ondisk_log_size":80528,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19298572,"kb_used_data":15485044,"kb_used_omap":1903,"kb_used_meta":3811600,"kb_avail":263816948,"statfs":{"total":289910292480,"available":270148554752,"internally_reserved":0,"allocated":15856685056,"data_stored":25731990376,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":3903078722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,7,5,5,14,22,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21815296,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5283,"num_read_kb":21056,"num_write":5327,"num_write_kb":21308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001955"},"pg_stats":[{"pgid":"2.7","version":"120'227390","reported_seq":452504,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:14.140486+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:07:14.140486+0000","last_peered":"2023-10-14T01:07:14.140486+0000","last_clean":"2023-10-14T01:07:14.140486+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:07:14.140486+0000","last_undegraded":"2023-10-14T01:07:14.140486+0000","last_fullsized":"2023-10-14T01:07:14.140486+0000","mapping_epoch":13,"log_start":"117'217311","ondisk_log_start":"117'217311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3147493559,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":225985,"num_read_kb":889412,"num_write":226334,"num_write_kb":905321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"120'238547","reported_seq":471776,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:14.145370+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:07:14.145370+0000","last_peered":"2023-10-14T01:07:14.145370+0000","last_clean":"2023-10-14T01:07:14.145370+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:07:14.145370+0000","last_undegraded":"2023-10-14T01:07:14.145370+0000","last_fullsized":"2023-10-14T01:07:14.145370+0000","mapping_epoch":13,"log_start":"117'228508","ondisk_log_start":"117'228508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3171397899,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":235901,"num_read_kb":906525,"num_write":226617,"num_write_kb":906067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"120'243718","reported_seq":484207,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:14.116641+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:07:14.116641+0000","last_peered":"2023-10-14T01:07:14.116641+0000","last_clean":"2023-10-14T01:07:14.116641+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:07:14.116641+0000","last_undegraded":"2023-10-14T01:07:14.116641+0000","last_fullsized":"2023-10-14T01:07:14.116641+0000","mapping_epoch":13,"log_start":"117'233712","ondisk_log_start":"117'233712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3334722091,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":241559,"num_read_kb":947022,"num_write":238936,"num_write_kb":955623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"120'257715","reported_seq":507037,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:14.141387+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:07:14.141387+0000","last_peered":"2023-10-14T01:07:14.141387+0000","last_clean":"2023-10-14T01:07:14.141387+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:07:14.141387+0000","last_undegraded":"2023-10-14T01:07:14.141387+0000","last_fullsized":"2023-10-14T01:07:14.141387+0000","mapping_epoch":13,"log_start":"118'247710","ondisk_log_start":"118'247710","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3426476463,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":250818,"num_read_kb":970671,"num_write":245790,"num_write_kb":982842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"120'248179","reported_seq":496242,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:09.698013+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:07:09.698013+0000","last_peered":"2023-10-14T01:07:09.698013+0000","last_clean":"2023-10-14T01:07:09.698013+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:07:09.698013+0000","last_undegraded":"2023-10-14T01:07:09.698013+0000","last_fullsized":"2023-10-14T01:07:09.698013+0000","mapping_epoch":13,"log_start":"117'238112","ondisk_log_start":"117'238112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":296,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 98s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3377107415,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":249353,"num_read_kb":957707,"num_write":242536,"num_write_kb":969637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"120'260198","reported_seq":517943,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:12.096615+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:07:12.096615+0000","last_peered":"2023-10-14T01:07:12.096615+0000","last_clean":"2023-10-14T01:07:12.096615+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:07:12.096615+0000","last_undegraded":"2023-10-14T01:07:12.096615+0000","last_fullsized":"2023-10-14T01:07:12.096615+0000","mapping_epoch":13,"log_start":"118'250111","ondisk_log_start":"118'250111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":287,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 100s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3584418212,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":258827,"num_read_kb":1010244,"num_write":255467,"num_write_kb":1021694,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"120'257654","reported_seq":511482,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:12.067147+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:07:12.067147+0000","last_peered":"2023-10-14T01:07:12.067147+0000","last_clean":"2023-10-14T01:07:12.067147+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:07:12.067147+0000","last_undegraded":"2023-10-14T01:07:12.067147+0000","last_fullsized":"2023-10-14T01:07:12.067147+0000","mapping_epoch":13,"log_start":"118'247613","ondisk_log_start":"118'247613","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":286,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 98s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3535802721,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":254981,"num_read_kb":995554,"num_write":252913,"num_write_kb":1011462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"120'262991","reported_seq":515259,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:14.138739+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:07:14.138739+0000","last_peered":"2023-10-14T01:07:14.138739+0000","last_clean":"2023-10-14T01:07:14.138739+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:07:14.138739+0000","last_undegraded":"2023-10-14T01:07:14.138739+0000","last_fullsized":"2023-10-14T01:07:14.138739+0000","mapping_epoch":13,"log_start":"118'252914","ondisk_log_start":"118'252914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3455259060,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":253773,"num_read_kb":981851,"num_write":247512,"num_write_kb":989691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":323,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:06:38.377423+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:06:38.377423+0000","last_peered":"2023-10-14T01:06:38.377423+0000","last_clean":"2023-10-14T01:06:38.377423+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:06:38.377423+0000","last_undegraded":"2023-10-14T01:06:38.377423+0000","last_fullsized":"2023-10-14T01:06:38.377423+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27032677420,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1971197,"num_read_kb":7658986,"num_write":1936105,"num_write_kb":7742337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15854043136,"data_stored":25730127960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80401,"ondisk_log_size":80401,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739177,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4256416,"kb_used_data":3023128,"kb_used_omap":349,"kb_used_meta":1232930,"kb_avail":90115424,"statfs":{"total":96636764160,"available":92278194176,"internally_reserved":0,"allocated":3095683072,"data_stored":5024072105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1262520657},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,2,4,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.96599999999999997,"15min":0.81999999999999995},"min":{"1min":0.21199999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":4.7850000000000001,"5min":16.474,"15min":16.474},"last":5.0270000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.76600000000000001,"15min":0.82699999999999996},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":4.6669999999999998,"5min":10.539999999999999,"15min":15.986000000000001},"last":4.9080000000000004}]},{"osd":1,"last update":"Sat Oct 14 01:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":1.5049999999999999,"15min":1.7629999999999999},"min":{"1min":0.27700000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.3499999999999996,"5min":13.519,"15min":14.303000000000001},"last":0.23599999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.5529999999999999,"15min":1.821},"min":{"1min":0.35299999999999998,"5min":0.27000000000000002,"15min":0.252},"max":{"1min":7.3970000000000002,"5min":13.565,"15min":14.35},"last":0.28799999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739170,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8705692,"kb_used_data":7745044,"kb_used_omap":961,"kb_used_meta":959678,"kb_avail":85666148,"statfs":{"total":96636764160,"available":87722135552,"internally_reserved":0,"allocated":7930925056,"data_stored":12868885308,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":982710671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,2,3,6,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0109999999999999,"15min":1.0169999999999999},"min":{"1min":0.24399999999999999,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":8.1500000000000004,"5min":9.548,"15min":14.773},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63300000000000001,"15min":0.85999999999999999},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":3.5270000000000001,"5min":9.593,"15min":10.161},"last":0.58899999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:06:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":0.77600000000000002,"15min":0.86099999999999999},"min":{"1min":0.27200000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":10.403,"5min":10.403,"15min":10.403},"last":0.62},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.91500000000000004,"15min":0.76100000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":4.1299999999999999,"5min":8.9179999999999993,"15min":9.2200000000000006},"last":4.532}]}]},{"osd":0,"up_from":8,"seq":34359739177,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6336464,"kb_used_data":4716872,"kb_used_omap":592,"kb_used_meta":1618991,"kb_avail":88035376,"statfs":{"total":96636764160,"available":90148225024,"internally_reserved":0,"allocated":4830076928,"data_stored":7839032963,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1657847394},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.66500000000000004,"15min":0.72799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":6.7480000000000002},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.99399999999999999,"15min":1.1439999999999999},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":3.6429999999999998,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.47899999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":1.2470000000000001,"15min":1.087},"min":{"1min":0.218,"5min":0.185,"15min":0.185},"max":{"1min":3.6989999999999998,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.316},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.0529999999999999,"15min":1.05},"min":{"1min":0.223,"5min":0.20999999999999999,"15min":0.182},"max":{"1min":3.681,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4829069312,"data_stored":7838107507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7929970688,"data_stored":12868013100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3095003136,"data_stored":5024007353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:07:16.338 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.338 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.339 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.339 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.340 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.340 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.340 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.341 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:16.341 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:07:18.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:18.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:18.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:18.316+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:19.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.204+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:07:19.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.216+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:19.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:19.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:19.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:19.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:19.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:19.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.232+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:19.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.232+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:19.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:19.232+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:20.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:20.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:20.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:20.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:20.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:20.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.956+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:20.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.956+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:20.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:20.988+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:21.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:21.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:21.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:21.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:21.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.056+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:21.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.056+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:21.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.476+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:21.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:21.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:21.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:21.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:21.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.504+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:21.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.504+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:21.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.504+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:21.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:21.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:21.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:21.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:21.904+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:22.415 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:22.416+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:07:22.441 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:22.440+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:07:22.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:22.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:22.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.540+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:22.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:22.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:22.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:22.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:22.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:22.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.840+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:22.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.880+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:22.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:22.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:22.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:22.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:22.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:22.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:24.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.700+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:24.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:24.758 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:24.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.796+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:24.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:24.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:24.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.800+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:24.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:24.800+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:25.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:25.476+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:25.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:25.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:25.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:25.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:25.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:25.540+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:29.994 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:29.991+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:07:29.994 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:29.991+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:07:31.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:31.659+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:31.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:31.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:31.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:31.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:31.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:31.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:31.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:31.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:31.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:31.691+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:32.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:32.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:32.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:32.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:32.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:32.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:32.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:32.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:32.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.711+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:07:32.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.711+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:07:32.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:32.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:32.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:32.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:32.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:32.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.755+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:32.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.755+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:32.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:32.787+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:33.523 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:33.523+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:07:33.523 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:33.523+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:07:34.279 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:34.279+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:07:34.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:34.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:34.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:34.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:34.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:34.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:34.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:34.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:34.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:34.727+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:35.590 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:35.587+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:07:35.605 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:35.599+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:07:36.055 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:07:36.055+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:07:36.343 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:07:37.260 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:07:37.261 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:07:37.296 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2158,"stamp":"2023-10-14T01:07:36.589374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27068001868,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1979654,"num_read_kb":7692255,"num_write":1944763,"num_write_kb":7778841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80507,"ondisk_log_size":80507,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19662160,"kb_used_data":15554292,"kb_used_omap":1903,"kb_used_meta":4105936,"kb_avail":263453360,"statfs":{"total":289910292480,"available":269776240640,"internally_reserved":0,"allocated":15927595008,"data_stored":25802900328,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":4204478786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,3,7,17,21,21,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18079744,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4286,"num_read_kb":17048,"num_write":4417,"num_write_kb":17668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004470"},"pg_stats":[{"pgid":"2.7","version":"120'228398","reported_seq":454442,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:34.632285+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:07:34.632285+0000","last_peered":"2023-10-14T01:07:34.632285+0000","last_clean":"2023-10-14T01:07:34.632285+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:07:34.632285+0000","last_undegraded":"2023-10-14T01:07:34.632285+0000","last_fullsized":"2023-10-14T01:07:34.632285+0000","mapping_epoch":13,"log_start":"117'218311","ondisk_log_start":"117'218311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3151610039,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":226914,"num_read_kb":893124,"num_write":227342,"num_write_kb":909353,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"120'239541","reported_seq":473747,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:34.610844+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:07:34.610844+0000","last_peered":"2023-10-14T01:07:34.610844+0000","last_clean":"2023-10-14T01:07:34.610844+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:07:34.610844+0000","last_undegraded":"2023-10-14T01:07:34.610844+0000","last_fullsized":"2023-10-14T01:07:34.610844+0000","mapping_epoch":13,"log_start":"118'229508","ondisk_log_start":"118'229508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3175207179,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":236864,"num_read_kb":910353,"num_write":227551,"num_write_kb":909803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"120'244796","reported_seq":486278,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:34.629858+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:07:34.629858+0000","last_peered":"2023-10-14T01:07:34.629858+0000","last_clean":"2023-10-14T01:07:34.629858+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:07:34.629858+0000","last_undegraded":"2023-10-14T01:07:34.629858+0000","last_fullsized":"2023-10-14T01:07:34.629858+0000","mapping_epoch":13,"log_start":"118'234712","ondisk_log_start":"118'234712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3339051563,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":242549,"num_read_kb":950950,"num_write":239994,"num_write_kb":959855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"120'258811","reported_seq":509149,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:34.624373+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:07:34.624373+0000","last_peered":"2023-10-14T01:07:34.624373+0000","last_clean":"2023-10-14T01:07:34.624373+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:07:34.624373+0000","last_undegraded":"2023-10-14T01:07:34.624373+0000","last_fullsized":"2023-10-14T01:07:34.624373+0000","mapping_epoch":13,"log_start":"118'248810","ondisk_log_start":"118'248810","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3430732207,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":251824,"num_read_kb":974679,"num_write":246826,"num_write_kb":986986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"120'249523","reported_seq":498890,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:34.947741+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:07:34.947741+0000","last_peered":"2023-10-14T01:07:34.947741+0000","last_clean":"2023-10-14T01:07:34.947741+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:07:34.947741+0000","last_undegraded":"2023-10-14T01:07:34.947741+0000","last_fullsized":"2023-10-14T01:07:34.947741+0000","mapping_epoch":13,"log_start":"118'239512","ondisk_log_start":"118'239512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":387,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 123s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3382301143,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":250621,"num_read_kb":962767,"num_write":243809,"num_write_kb":974729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"120'261364","reported_seq":520208,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:32.281998+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:07:32.281998+0000","last_peered":"2023-10-14T01:07:32.281998+0000","last_clean":"2023-10-14T01:07:32.281998+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:07:32.281998+0000","last_undegraded":"2023-10-14T01:07:32.281998+0000","last_fullsized":"2023-10-14T01:07:32.281998+0000","mapping_epoch":13,"log_start":"118'251311","ondisk_log_start":"118'251311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":352,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 120s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3588993444,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":259912,"num_read_kb":1014552,"num_write":256584,"num_write_kb":1026162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"120'258737","reported_seq":513680,"reported_epoch":120,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:32.311148+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:07:32.311148+0000","last_peered":"2023-10-14T01:07:32.311148+0000","last_clean":"2023-10-14T01:07:32.311148+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:07:32.311148+0000","last_undegraded":"2023-10-14T01:07:32.311148+0000","last_fullsized":"2023-10-14T01:07:32.311148+0000","mapping_epoch":13,"log_start":"118'248713","ondisk_log_start":"118'248713","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":355,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 118s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3540046177,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":256069,"num_read_kb":999882,"num_write":253945,"num_write_kb":1015590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"120'264101","reported_seq":517387,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:34.626086+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:07:34.626086+0000","last_peered":"2023-10-14T01:07:34.626086+0000","last_clean":"2023-10-14T01:07:34.626086+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:07:34.626086+0000","last_undegraded":"2023-10-14T01:07:34.626086+0000","last_fullsized":"2023-10-14T01:07:34.626086+0000","mapping_epoch":13,"log_start":"118'254014","ondisk_log_start":"118'254014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3459469748,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":254775,"num_read_kb":985839,"num_write":248543,"num_write_kb":993815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":323,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:06:38.377423+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:06:38.377423+0000","last_peered":"2023-10-14T01:06:38.377423+0000","last_clean":"2023-10-14T01:06:38.377423+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:06:38.377423+0000","last_undegraded":"2023-10-14T01:06:38.377423+0000","last_fullsized":"2023-10-14T01:06:38.377423+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27067411500,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1979528,"num_read_kb":7692146,"num_write":1944594,"num_write_kb":7776293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15926558720,"data_stored":25802643544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80380,"ondisk_log_size":80380,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739181,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4336888,"kb_used_data":3035568,"kb_used_omap":349,"kb_used_meta":1300962,"kb_avail":90034952,"statfs":{"total":96636764160,"available":92195790848,"internally_reserved":0,"allocated":3108421632,"data_stored":5036810665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1332185425},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,3,9,10,11,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.96599999999999997,"15min":0.81999999999999995},"min":{"1min":0.21199999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":4.7850000000000001,"5min":16.474,"15min":16.474},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.76600000000000001,"15min":0.82699999999999996},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":4.6669999999999998,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.27100000000000002}]},{"osd":1,"last update":"Sat Oct 14 01:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":1.5049999999999999,"15min":1.7629999999999999},"min":{"1min":0.27700000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.3499999999999996,"5min":13.519,"15min":14.303000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.774,"5min":1.5529999999999999,"15min":1.821},"min":{"1min":0.35299999999999998,"5min":0.27000000000000002,"15min":0.252},"max":{"1min":7.3970000000000002,"5min":13.565,"15min":14.35},"last":0.83699999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739174,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8861404,"kb_used_data":7777104,"kb_used_omap":961,"kb_used_meta":1083326,"kb_avail":85510436,"statfs":{"total":96636764160,"available":87562686464,"internally_reserved":0,"allocated":7963754496,"data_stored":12901714748,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1109326223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,0,4,6,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:06:38 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.0109999999999999,"15min":1.0169999999999999},"min":{"1min":0.24399999999999999,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":8.1500000000000004,"5min":9.548,"15min":14.773},"last":0.251},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.63300000000000001,"15min":0.85999999999999999},"min":{"1min":0.24399999999999999,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":3.5270000000000001,"5min":9.593,"15min":10.161},"last":0.42599999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:06:38 2023","interfaces":[{"interface":"back","average":{"1min":1.5509999999999999,"5min":0.77600000000000002,"15min":0.86099999999999999},"min":{"1min":0.27200000000000002,"5min":0.20999999999999999,"15min":0.20899999999999999},"max":{"1min":10.403,"5min":10.403,"15min":10.403},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.91500000000000004,"15min":0.76100000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":4.1299999999999999,"5min":8.9179999999999993,"15min":9.2200000000000006},"last":0.309}]}]},{"osd":0,"up_from":8,"seq":34359739182,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6463868,"kb_used_data":4741620,"kb_used_omap":592,"kb_used_meta":1721647,"kb_avail":87907972,"statfs":{"total":96636764160,"available":90017763328,"internally_reserved":0,"allocated":4855418880,"data_stored":7864374915,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1762967138},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,9,5,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.66500000000000004,"15min":0.72799999999999998},"min":{"1min":0.182,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":5.8220000000000001,"5min":5.8220000000000001,"15min":6.7480000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.99399999999999999,"15min":1.1439999999999999},"min":{"1min":0.214,"5min":0.191,"15min":0.191},"max":{"1min":3.6429999999999998,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.33000000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:06:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":1.2470000000000001,"15min":1.087},"min":{"1min":0.218,"5min":0.185,"15min":0.185},"max":{"1min":3.6989999999999998,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":1.0529999999999999,"15min":1.05},"min":{"1min":0.223,"5min":0.20999999999999999,"15min":0.182},"max":{"1min":3.681,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.28000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4854722560,"data_stored":7863760755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7963856896,"data_stored":12901899308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3107979264,"data_stored":5036983481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:07:37.298 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.298 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.299 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.299 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.299 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.300 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.300 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.300 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:37.301 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:07:37.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:37.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:37.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:37.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:37.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:37.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:37.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:37.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:37.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:37.527+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:37.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:37.539+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:40.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.567+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:40.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:40.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:40.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:40.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:40.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:40.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.707+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:40.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.707+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:40.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:40.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:41.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:41.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:41.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:41.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:41.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:41.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:41.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:41.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:41.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:41.059+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:41.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:41.059+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:41.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:41.059+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:42.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.659+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:42.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:42.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:42.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:42.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:42.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:42.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.695+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:42.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.695+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:42.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:42.695+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:43.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.419+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:43.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.419+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:43.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:43.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:43.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:43.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.455+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:43.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.455+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:43.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:43.455+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:44.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:44.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:44.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:44.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:44.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:44.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.187+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:44.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.187+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:44.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:44.195+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:46.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:46.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:46.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:46.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:46.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:46.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:46.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:46.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:46.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:46.507+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:46.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:46.507+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:46.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:46.535+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:47.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.131+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:47.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:47.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:47.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:47.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:47.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:47.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.199+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:47.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.199+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:47.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:47.203+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:48.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.071+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:48.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:48.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:48.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:48.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.139+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:48.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.139+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:48.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.139+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:48.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.783+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:48.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:48.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:48.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:48.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:48.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:48.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.835+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:48.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.835+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:48.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:48.895+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:49.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.815+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:49.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:49.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:49.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:49.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:49.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:49.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.927+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:49.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.927+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:49.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:49.935+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:53.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:53.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:53.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:53.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:53.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:53.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:53.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:53.615+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:53.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:53.615+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:53.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:53.615+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:53.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:53.615+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:54.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.267+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:54.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:54.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:54.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:54.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:54.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.315+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:54.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.315+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:54.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:54.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:54.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:54.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:54.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.579+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:54.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.579+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:54.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:54.579+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:56.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.635+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:56.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:56.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:56.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:56.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:56.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:56.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.655+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:56.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.655+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:56.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:56.687+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:57.302 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:07:57.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:57.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:57.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:57.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:57.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.659+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:57.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.659+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:57.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.659+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:57.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:57.659+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:58.031 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:07:58.032 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:07:58.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.031+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:07:58.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:58.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:58.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:58.093 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2169,"stamp":"2023-10-14T01:07:56.597192+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27101654604,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1987581,"num_read_kb":7723815,"num_write":1953020,"num_write_kb":7811869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80402,"ondisk_log_size":80402,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":19992168,"kb_used_data":15621264,"kb_used_omap":1903,"kb_used_meta":4368976,"kb_avail":263123352,"statfs":{"total":289910292480,"available":269438312448,"internally_reserved":0,"allocated":15996174336,"data_stored":25871467959,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":4473831746},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,3,6,7,18,12,13,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16855040,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3947,"num_read_kb":15712,"num_write":4131,"num_write_kb":16524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004099"},"pg_stats":[{"pgid":"2.7","version":"121'229371","reported_seq":456353,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:07:55.130357+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:07:55.130357+0000","last_peered":"2023-10-14T01:07:55.130357+0000","last_clean":"2023-10-14T01:07:55.130357+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:07:55.130357+0000","last_undegraded":"2023-10-14T01:07:55.130357+0000","last_fullsized":"2023-10-14T01:07:55.130357+0000","mapping_epoch":13,"log_start":"118'219311","ondisk_log_start":"118'219311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3155583159,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":227853,"num_read_kb":896868,"num_write":228315,"num_write_kb":913245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"121'240552","reported_seq":475734,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:07:55.067548+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:07:55.067548+0000","last_peered":"2023-10-14T01:07:55.067548+0000","last_clean":"2023-10-14T01:07:55.067548+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:07:55.067548+0000","last_undegraded":"2023-10-14T01:07:55.067548+0000","last_fullsized":"2023-10-14T01:07:55.067548+0000","mapping_epoch":13,"log_start":"118'230508","ondisk_log_start":"118'230508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3179122955,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":237828,"num_read_kb":914197,"num_write":228517,"num_write_kb":913667,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"121'245823","reported_seq":488290,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:07:55.067991+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:07:55.067991+0000","last_peered":"2023-10-14T01:07:55.067991+0000","last_clean":"2023-10-14T01:07:55.067991+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:07:55.067991+0000","last_undegraded":"2023-10-14T01:07:55.067991+0000","last_fullsized":"2023-10-14T01:07:55.067991+0000","mapping_epoch":13,"log_start":"118'235812","ondisk_log_start":"118'235812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3343176235,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":243528,"num_read_kb":954858,"num_write":241006,"num_write_kb":963903,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"121'259928","reported_seq":511333,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:07:55.051299+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:07:55.051299+0000","last_peered":"2023-10-14T01:07:55.051299+0000","last_clean":"2023-10-14T01:07:55.051299+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:07:55.051299+0000","last_undegraded":"2023-10-14T01:07:55.051299+0000","last_fullsized":"2023-10-14T01:07:55.051299+0000","mapping_epoch":13,"log_start":"118'249910","ondisk_log_start":"118'249910","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3435106735,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":252877,"num_read_kb":978875,"num_write":247898,"num_write_kb":991274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"121'250631","reported_seq":500982,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:55.129397+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:07:55.129397+0000","last_peered":"2023-10-14T01:07:55.129397+0000","last_clean":"2023-10-14T01:07:55.129397+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:07:55.129397+0000","last_undegraded":"2023-10-14T01:07:55.129397+0000","last_fullsized":"2023-10-14T01:07:55.129397+0000","mapping_epoch":13,"log_start":"118'240612","ondisk_log_start":"118'240612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":452,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3386601943,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":251578,"num_read_kb":966571,"num_write":244865,"num_write_kb":978953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"121'262514","reported_seq":522426,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:52.609625+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:07:52.609625+0000","last_peered":"2023-10-14T01:07:52.609625+0000","last_clean":"2023-10-14T01:07:52.609625+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:07:52.609625+0000","last_undegraded":"2023-10-14T01:07:52.609625+0000","last_fullsized":"2023-10-14T01:07:52.609625+0000","mapping_epoch":13,"log_start":"118'252511","ondisk_log_start":"118'252511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":402,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 140s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3593482660,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":260964,"num_read_kb":1018740,"num_write":257681,"num_write_kb":1030550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"121'259873","reported_seq":515836,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:07:52.584129+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:07:52.584129+0000","last_peered":"2023-10-14T01:07:52.584129+0000","last_clean":"2023-10-14T01:07:52.584129+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:07:52.584129+0000","last_undegraded":"2023-10-14T01:07:52.584129+0000","last_fullsized":"2023-10-14T01:07:52.584129+0000","mapping_epoch":13,"log_start":"118'249814","ondisk_log_start":"118'249814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":436,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 138s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3544379745,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":257072,"num_read_kb":1003866,"num_write":255012,"num_write_kb":1019858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"121'265175","reported_seq":519459,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:07:55.045191+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:07:55.045191+0000","last_peered":"2023-10-14T01:07:55.045191+0000","last_clean":"2023-10-14T01:07:55.045191+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:07:55.045191+0000","last_undegraded":"2023-10-14T01:07:55.045191+0000","last_fullsized":"2023-10-14T01:07:55.045191+0000","mapping_epoch":13,"log_start":"118'255114","ondisk_log_start":"118'255114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3463610804,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":255755,"num_read_kb":989731,"num_write":249557,"num_write_kb":997871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":324,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:09.042467+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:07:09.042467+0000","last_peered":"2023-10-14T01:07:09.042467+0000","last_clean":"2023-10-14T01:07:09.042467+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:07:09.042467+0000","last_undegraded":"2023-10-14T01:07:09.042467+0000","last_fullsized":"2023-10-14T01:07:09.042467+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27101064236,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1987455,"num_read_kb":7723706,"num_write":1952851,"num_write_kb":7809321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15994040320,"data_stored":25870125144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80275,"ondisk_log_size":80275,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739185,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4398268,"kb_used_data":3048372,"kb_used_omap":349,"kb_used_meta":1349538,"kb_avail":89973572,"statfs":{"total":96636764160,"available":92132937728,"internally_reserved":0,"allocated":3121532928,"data_stored":5049918062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1381927249},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,0,1,6,1,2,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.96599999999999997,"15min":0.81999999999999995},"min":{"1min":0.21199999999999999,"5min":0.17299999999999999,"15min":0.17299999999999999},"max":{"1min":4.7850000000000001,"5min":16.474,"15min":16.474},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.76600000000000001,"15min":0.82699999999999996},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":4.6669999999999998,"5min":10.539999999999999,"15min":15.986000000000001},"last":0.35899999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:06:59 2023","interfaces":[{"interface":"back","average":{"1min":1.716,"5min":1.5049999999999999,"15min":1.7629999999999999},"min":{"1min":0.27700000000000002,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":7.3499999999999996,"5min":13.519,"15min":14.303000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":1.774,"5min":1.5529999999999999,"15min":1.821},"min":{"1min":0.35299999999999998,"5min":0.27000000000000002,"15min":0.252},"max":{"1min":7.3970000000000002,"5min":13.565,"15min":14.35},"last":0.39900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739178,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9025404,"kb_used_data":7811188,"kb_used_omap":961,"kb_used_meta":1213246,"kb_avail":85346436,"statfs":{"total":96636764160,"available":87394750464,"internally_reserved":0,"allocated":7998656512,"data_stored":12936612865,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1242364303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,6,4,9,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.93100000000000005,"15min":0.99299999999999999},"min":{"1min":0.19800000000000001,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":1.1639999999999999,"5min":9.548,"15min":14.773},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.82499999999999996},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":1.913,"5min":9.593,"15min":10.161},"last":0.38800000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.76300000000000001,"15min":0.80100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.20899999999999999},"max":{"1min":1.5640000000000001,"5min":10.403,"15min":10.403},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.81699999999999995,"15min":0.75900000000000001},"min":{"1min":0.24399999999999999,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":4.532,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":0.40100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739186,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6568496,"kb_used_data":4761704,"kb_used_omap":592,"kb_used_meta":1806191,"kb_avail":87803344,"statfs":{"total":96636764160,"available":89910624256,"internally_reserved":0,"allocated":4875984896,"data_stored":7884937032,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1849540194},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.19900000000000001,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":1.5209999999999999,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.90500000000000003,"15min":1.077},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.191},"max":{"1min":1.5409999999999999,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.35199999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.21,"15min":1.1040000000000001},"min":{"1min":0.222,"5min":0.20799999999999999,"15min":0.185},"max":{"1min":4.5990000000000002,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.98599999999999999,"15min":1.0369999999999999},"min":{"1min":0.23899999999999999,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":3.302,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.38700000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4875264000,"data_stored":7884302195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":7997874176,"data_stored":12935916588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3120902144,"data_stored":5049906361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:07:58.095 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.095 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.095 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.096 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.096 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.097 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.097 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.098 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:07:58.098 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:07:58.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:58.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.083+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:58.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.083+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:58.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.103+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:58.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:58.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:58.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:58.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.175+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:58.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.187+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:58.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:58.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.551+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:58.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:07:58.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:07:58.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:07:58.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.611+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:07:58.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.611+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:07:58.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.627+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:07:58.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:07:58.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:07:58.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:07:58.733 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:07:58.787 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:07:59.563 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:07:59.603 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:08:00.281 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:08:00.377 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:08:01.171 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:08:01.211 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-14T01:08:01.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:01.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:01.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:01.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:01.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:01.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:01.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:01.691+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:01.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:01.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:01.909 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:01.907+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:01.991 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-14T01:08:02.039 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.1 2023-10-14T01:08:02.895 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.1 on osd.2 to deep-scrub 2023-10-14T01:08:02.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.0 2023-10-14T01:08:03.768 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.0 on osd.2 to deep-scrub 2023-10-14T01:08:03.839 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:08:04.694 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:08:04.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:04.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:04.752 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:08:04.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:04.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:04.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:04.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:04.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:04.786+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:04.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:04.786+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:04.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:04.802+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:05.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.430+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:08:05.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.430+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:08:05.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.438+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:05.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:05.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:05.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:05.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:05.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:05.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.474+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:05.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.474+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:05.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:05.474+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:08.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.014+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:08.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:08.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:08.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:08.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:08.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:08.072 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.062+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:08.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.078+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:08.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.078+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:08.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:08.654+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:11.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:11.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:11.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:11.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.434+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:11.710 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:11.706+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:08:11.751 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:11.750+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:11.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:11.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:11.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:11.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:11.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:11.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.970+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:11.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.970+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:11.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:11.970+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:16.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.046+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:08:16.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.062+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:08:16.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.066+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:16.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.074+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:16.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:16.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:16.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:16.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:16.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:16.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:16.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:16.102+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:18.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.218+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:08:18.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.246+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:18.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:18.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:18.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.302+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:18.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:18.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:18.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.314+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:18.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.382+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:18.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.382+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:18.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.814+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:18.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:18.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:18.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:18.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:18.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.846+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:18.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.846+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:18.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:18.854+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:19.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:19.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:19.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:19.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:19.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:19.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:19.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:19.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:19.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:19.238+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:19.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:19.238+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:19.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:19.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:20.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:20.990+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:21.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:21.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:21.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:21.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:21.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:21.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.046+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:21.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.046+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:21.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.046+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:21.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.414+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:21.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:21.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:21.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:21.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:21.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:21.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.434+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:21.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.434+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:21.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:21.434+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:22.902 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:22.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:22.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:22.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:22.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:22.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.946+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:22.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.946+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:22.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:22.950+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:23.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:23.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:23.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:23.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:23.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.226+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:23.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:23.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:23.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.370+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:23.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.370+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:23.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.370+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:23.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:23.374+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:24.462 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:24.458+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:24.753 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:08:25.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.294+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:25.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:25.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:25.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:25.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:25.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:25.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:25.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:25.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:25.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:25.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:25.529 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:08:25.530 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:08:25.562 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2183,"stamp":"2023-10-14T01:08:24.602935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27145510476,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":1998016,"num_read_kb":7765435,"num_write":1963756,"num_write_kb":7854813,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80583,"ondisk_log_size":80583,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":20330604,"kb_used_data":15704084,"kb_used_omap":1903,"kb_used_meta":4624592,"kb_avail":262784916,"statfs":{"total":289910292480,"available":269091753984,"internally_reserved":0,"allocated":16080982016,"data_stored":25956275639,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":4735582530},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,8,9,19,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19533824,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4730,"num_read_kb":18852,"num_write":4780,"num_write_kb":19120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001931"},"pg_stats":[{"pgid":"2.7","version":"121'230536","reported_seq":458688,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:20.846749+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:08:20.846749+0000","last_peered":"2023-10-14T01:08:20.846749+0000","last_clean":"2023-10-14T01:08:20.846749+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:08:20.846749+0000","last_undegraded":"2023-10-14T01:08:20.846749+0000","last_fullsized":"2023-10-14T01:08:20.846749+0000","mapping_epoch":13,"log_start":"118'220511","ondisk_log_start":"118'220511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3160330423,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":229019,"num_read_kb":901520,"num_write":229480,"num_write_kb":917905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"121'241849","reported_seq":478175,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:20.847131+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:08:20.847131+0000","last_peered":"2023-10-14T01:08:20.847131+0000","last_clean":"2023-10-14T01:08:20.847131+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:08:20.847131+0000","last_undegraded":"2023-10-14T01:08:20.847131+0000","last_fullsized":"2023-10-14T01:08:20.847131+0000","mapping_epoch":13,"log_start":"118'231808","ondisk_log_start":"118'231808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3184234763,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":238954,"num_read_kb":918681,"num_write":229766,"num_write_kb":918663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"121'247096","reported_seq":490802,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:20.844906+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:08:20.844906+0000","last_peered":"2023-10-14T01:08:20.844906+0000","last_clean":"2023-10-14T01:08:20.844906+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:08:20.844906+0000","last_undegraded":"2023-10-14T01:08:20.844906+0000","last_fullsized":"2023-10-14T01:08:20.844906+0000","mapping_epoch":13,"log_start":"118'237012","ondisk_log_start":"118'237012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3348292139,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":244761,"num_read_kb":959782,"num_write":242262,"num_write_kb":968927,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"121'261288","reported_seq":514035,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:20.865016+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:08:20.865016+0000","last_peered":"2023-10-14T01:08:20.865016+0000","last_clean":"2023-10-14T01:08:20.865016+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:08:20.865016+0000","last_undegraded":"2023-10-14T01:08:20.865016+0000","last_fullsized":"2023-10-14T01:08:20.865016+0000","mapping_epoch":13,"log_start":"118'251210","ondisk_log_start":"118'251210","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3440468399,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":254202,"num_read_kb":984159,"num_write":249209,"num_write_kb":996518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"121'251992","reported_seq":503580,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:08:20.362411+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:08:20.362411+0000","last_peered":"2023-10-14T01:08:20.362411+0000","last_clean":"2023-10-14T01:08:20.362411+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:08:20.362411+0000","last_undegraded":"2023-10-14T01:08:20.362411+0000","last_fullsized":"2023-10-14T01:08:20.362411+0000","mapping_epoch":13,"log_start":"118'241912","ondisk_log_start":"118'241912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":522,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 169s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3391816151,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":252774,"num_read_kb":971339,"num_write":246133,"num_write_kb":984025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"121'264172","reported_seq":525636,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:08:23.076393+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:08:23.076393+0000","last_peered":"2023-10-14T01:08:23.076393+0000","last_clean":"2023-10-14T01:08:23.076393+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:08:23.076393+0000","last_undegraded":"2023-10-14T01:08:23.076393+0000","last_fullsized":"2023-10-14T01:08:23.076393+0000","mapping_epoch":13,"log_start":"118'254111","ondisk_log_start":"118'254111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":494,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 170s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3599925668,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":262490,"num_read_kb":1024836,"num_write":259262,"num_write_kb":1036874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"121'261550","reported_seq":519060,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:08:23.077813+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:08:23.077813+0000","last_peered":"2023-10-14T01:08:23.077813+0000","last_clean":"2023-10-14T01:08:23.077813+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:08:23.077813+0000","last_undegraded":"2023-10-14T01:08:23.077813+0000","last_fullsized":"2023-10-14T01:08:23.077813+0000","mapping_epoch":13,"log_start":"118'251514","ondisk_log_start":"118'251514","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":556,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 169s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3550855521,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":258592,"num_read_kb":1009926,"num_write":256592,"num_write_kb":1026178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"121'266565","reported_seq":522210,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:20.814867+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:08:20.814867+0000","last_peered":"2023-10-14T01:08:20.814867+0000","last_clean":"2023-10-14T01:08:20.814867+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:08:20.814867+0000","last_undegraded":"2023-10-14T01:08:20.814867+0000","last_fullsized":"2023-10-14T01:08:20.814867+0000","mapping_epoch":13,"log_start":"118'256514","ondisk_log_start":"118'256514","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3468997044,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":257098,"num_read_kb":995083,"num_write":250883,"num_write_kb":1003175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":324,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:09.042467+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:07:09.042467+0000","last_peered":"2023-10-14T01:07:09.042467+0000","last_clean":"2023-10-14T01:07:09.042467+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:07:09.042467+0000","last_undegraded":"2023-10-14T01:07:09.042467+0000","last_fullsized":"2023-10-14T01:07:09.042467+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27144920108,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":1997890,"num_read_kb":7765326,"num_write":1963587,"num_write_kb":7852265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16080412672,"data_stored":25956497496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80456,"ondisk_log_size":80456,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739191,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4476624,"kb_used_data":3067080,"kb_used_omap":349,"kb_used_meta":1409186,"kb_avail":89895216,"statfs":{"total":96636764160,"available":92052701184,"internally_reserved":0,"allocated":3140689920,"data_stored":5069075054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1443006801},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.95499999999999996,"15min":0.80200000000000005},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.17299999999999999},"max":{"1min":5.1210000000000004,"5min":7.6390000000000002,"15min":16.474},"last":0.248},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.0269999999999999,"15min":0.89400000000000002},"min":{"1min":0.217,"5min":0.187,"15min":0.187},"max":{"1min":11.44,"5min":11.44,"15min":15.986000000000001},"last":1.1579999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.4970000000000001,"15min":1.74},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.20200000000000001},"max":{"1min":4.5069999999999997,"5min":13.519,"15min":14.303000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.121,"5min":1.544,"15min":1.798},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":4.5359999999999996,"5min":13.565,"15min":14.35},"last":0.81599999999999995}]}]},{"osd":1,"up_from":8,"seq":34359739183,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9197128,"kb_used_data":7850816,"kb_used_omap":961,"kb_used_meta":1345342,"kb_avail":85174712,"statfs":{"total":96636764160,"available":87218905088,"internally_reserved":0,"allocated":8039235584,"data_stored":12977191937,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1377630607},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.93100000000000005,"15min":0.99299999999999999},"min":{"1min":0.19800000000000001,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":1.1639999999999999,"5min":9.548,"15min":14.773},"last":14.957000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.82499999999999996},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":1.913,"5min":9.593,"15min":10.161},"last":0.73299999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.76300000000000001,"15min":0.80100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.20899999999999999},"max":{"1min":1.5640000000000001,"5min":10.403,"15min":10.403},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.81699999999999995,"15min":0.75900000000000001},"min":{"1min":0.24399999999999999,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":4.532,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":0.68799999999999994}]}]},{"osd":0,"up_from":8,"seq":34359739191,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6656852,"kb_used_data":4786188,"kb_used_omap":592,"kb_used_meta":1870063,"kb_avail":87714988,"statfs":{"total":96636764160,"available":89820147712,"internally_reserved":0,"allocated":4901056512,"data_stored":7910008648,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1914945122},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.19900000000000001,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":1.5209999999999999,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.90500000000000003,"15min":1.077},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.191},"max":{"1min":1.5409999999999999,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.22700000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.21,"15min":1.1040000000000001},"min":{"1min":0.222,"5min":0.20799999999999999,"15min":0.185},"max":{"1min":4.5990000000000002,"5min":12.622999999999999,"15min":12.622999999999999},"last":5.1079999999999997},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.98599999999999999,"15min":1.0369999999999999},"min":{"1min":0.23899999999999999,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":3.302,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.26900000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4900392960,"data_stored":7909431155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8039768064,"data_stored":12977810476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3140251648,"data_stored":5069255865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:08:25.563 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.563 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.564 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.564 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.564 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.565 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.565 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.565 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:25.566 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:08:27.325 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:27.322+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:08:27.345 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:27.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:08:27.364 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:27.362+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:29.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:29.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:29.875 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:29.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:29.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:29.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:29.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:29.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:29.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:29.906+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:32.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.938+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:32.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:32.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:32.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:32.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:32.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.982+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:32.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.982+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:32.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:32.982+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:37.764 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:37.762+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:08:37.777 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:37.774+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:40.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.009+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:40.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.025+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:40.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:40.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:40.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.037+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:40.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:40.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:40.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:40.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.085+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:40.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:40.141+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:43.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:43.292 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:43.289+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:08:43.303 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:43.301+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:43.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:43.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:43.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:43.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:43.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:43.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:43.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:43.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:45.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:45.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:45.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:45.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:45.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:45.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:45.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:45.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:45.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:45.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:45.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:45.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:45.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:45.513+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:45.567 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:08:46.044 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:46.041+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:46.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:46.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:46.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:46.448 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:08:46.448 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:08:46.514 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2193,"stamp":"2023-10-14T01:08:44.609438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27179748940,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2006157,"num_read_kb":7797927,"num_write":1972125,"num_write_kb":7888289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80620,"ondisk_log_size":80620,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":20679172,"kb_used_data":15771684,"kb_used_omap":1903,"kb_used_meta":4905552,"kb_avail":262436348,"statfs":{"total":289910292480,"available":268734820352,"internally_reserved":0,"allocated":16150204416,"data_stored":26025498039,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":5023285570},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,9,12,18,15,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19525632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4601,"num_read_kb":18364,"num_write":4763,"num_write_kb":19052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002843"},"pg_stats":[{"pgid":"2.7","version":"121'231512","reported_seq":460588,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:41.420322+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:08:41.420322+0000","last_peered":"2023-10-14T01:08:41.420322+0000","last_clean":"2023-10-14T01:08:41.420322+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:08:41.420322+0000","last_undegraded":"2023-10-14T01:08:41.420322+0000","last_fullsized":"2023-10-14T01:08:41.420322+0000","mapping_epoch":13,"log_start":"118'221511","ondisk_log_start":"118'221511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3164328119,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":229945,"num_read_kb":905220,"num_write":230456,"num_write_kb":921809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"121'242896","reported_seq":480168,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:41.408175+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:08:41.408175+0000","last_peered":"2023-10-14T01:08:41.408175+0000","last_clean":"2023-10-14T01:08:41.408175+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:08:41.408175+0000","last_undegraded":"2023-10-14T01:08:41.408175+0000","last_fullsized":"2023-10-14T01:08:41.408175+0000","mapping_epoch":13,"log_start":"118'232808","ondisk_log_start":"118'232808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3188363531,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":239888,"num_read_kb":922417,"num_write":230777,"num_write_kb":922707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"121'248181","reported_seq":492932,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:41.386399+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:08:41.386399+0000","last_peered":"2023-10-14T01:08:41.386399+0000","last_clean":"2023-10-14T01:08:41.386399+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:08:41.386399+0000","last_undegraded":"2023-10-14T01:08:41.386399+0000","last_fullsized":"2023-10-14T01:08:41.386399+0000","mapping_epoch":13,"log_start":"118'238112","ondisk_log_start":"118'238112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3352691243,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":245801,"num_read_kb":963914,"num_write":243336,"num_write_kb":973223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"121'262374","reported_seq":516188,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:41.448127+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:08:41.448127+0000","last_peered":"2023-10-14T01:08:41.448127+0000","last_clean":"2023-10-14T01:08:41.448127+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:08:41.448127+0000","last_undegraded":"2023-10-14T01:08:41.448127+0000","last_fullsized":"2023-10-14T01:08:41.448127+0000","mapping_epoch":13,"log_start":"119'252310","ondisk_log_start":"119'252310","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3444769199,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":255259,"num_read_kb":988379,"num_write":250264,"num_write_kb":1000738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"121'253042","reported_seq":505654,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:08:40.480104+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:08:40.480104+0000","last_peered":"2023-10-14T01:08:40.480104+0000","last_clean":"2023-10-14T01:08:40.480104+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:08:40.480104+0000","last_undegraded":"2023-10-14T01:08:40.480104+0000","last_fullsized":"2023-10-14T01:08:40.480104+0000","mapping_epoch":13,"log_start":"119'243012","ondisk_log_start":"119'243012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":601,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 189s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3395846615,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":253773,"num_read_kb":975323,"num_write":247116,"num_write_kb":987957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"121'265307","reported_seq":527875,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:08:43.245437+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:08:43.245437+0000","last_peered":"2023-10-14T01:08:43.245437+0000","last_clean":"2023-10-14T01:08:43.245437+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:08:43.245437+0000","last_undegraded":"2023-10-14T01:08:43.245437+0000","last_fullsized":"2023-10-14T01:08:43.245437+0000","mapping_epoch":13,"log_start":"119'255211","ondisk_log_start":"119'255211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":529,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 191s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3604435364,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":263584,"num_read_kb":1029208,"num_write":260362,"num_write_kb":1041274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"121'262714","reported_seq":521277,"reported_epoch":121,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:08:43.242000+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:08:43.242000+0000","last_peered":"2023-10-14T01:08:43.242000+0000","last_clean":"2023-10-14T01:08:43.242000+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:08:43.242000+0000","last_undegraded":"2023-10-14T01:08:43.242000+0000","last_fullsized":"2023-10-14T01:08:43.242000+0000","mapping_epoch":13,"log_start":"119'252614","ondisk_log_start":"119'252614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":607,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 189s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3555443041,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":259628,"num_read_kb":1014062,"num_write":257713,"num_write_kb":1030662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"121'267659","reported_seq":524376,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:08:41.446094+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:08:41.446094+0000","last_peered":"2023-10-14T01:08:41.446094+0000","last_clean":"2023-10-14T01:08:41.446094+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:08:41.446094+0000","last_undegraded":"2023-10-14T01:08:41.446094+0000","last_fullsized":"2023-10-14T01:08:41.446094+0000","mapping_epoch":13,"log_start":"119'257614","ondisk_log_start":"119'257614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3473281460,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":258153,"num_read_kb":999295,"num_write":251932,"num_write_kb":1007371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":324,"reported_epoch":120,"state":"active+clean","last_fresh":"2023-10-14T01:07:09.042467+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:07:09.042467+0000","last_peered":"2023-10-14T01:07:09.042467+0000","last_clean":"2023-10-14T01:07:09.042467+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:07:09.042467+0000","last_undegraded":"2023-10-14T01:07:09.042467+0000","last_fullsized":"2023-10-14T01:07:09.042467+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27179158572,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2006031,"num_read_kb":7797818,"num_write":1971956,"num_write_kb":7885741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16148987904,"data_stored":26025072728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80493,"ondisk_log_size":80493,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739195,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4562388,"kb_used_data":3080140,"kb_used_omap":349,"kb_used_meta":1481890,"kb_avail":89809452,"statfs":{"total":96636764160,"available":91964878848,"internally_reserved":0,"allocated":3154063360,"data_stored":5082448494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1517455697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,6,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.95499999999999996,"15min":0.80200000000000005},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.17299999999999999},"max":{"1min":5.1210000000000004,"5min":7.6390000000000002,"15min":16.474},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.0269999999999999,"15min":0.89400000000000002},"min":{"1min":0.217,"5min":0.187,"15min":0.187},"max":{"1min":11.44,"5min":11.44,"15min":15.986000000000001},"last":2.29}]},{"osd":1,"last update":"Sat Oct 14 01:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.4970000000000001,"15min":1.74},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.20200000000000001},"max":{"1min":4.5069999999999997,"5min":13.519,"15min":14.303000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.544,"15min":1.798},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":4.5359999999999996,"5min":13.565,"15min":14.35},"last":0.42999999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739187,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9385540,"kb_used_data":7885112,"kb_used_omap":961,"kb_used_meta":1499454,"kb_avail":84986300,"statfs":{"total":96636764160,"available":87025971200,"internally_reserved":0,"allocated":8074354688,"data_stored":13012311041,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1535441295},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,3,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.93100000000000005,"15min":0.99299999999999999},"min":{"1min":0.19800000000000001,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":1.1639999999999999,"5min":9.548,"15min":14.773},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.60799999999999998,"15min":0.82499999999999996},"min":{"1min":0.26000000000000001,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":1.913,"5min":9.593,"15min":10.161},"last":0.40999999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:07:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.76300000000000001,"15min":0.80100000000000005},"min":{"1min":0.23899999999999999,"5min":0.23599999999999999,"15min":0.20899999999999999},"max":{"1min":1.5640000000000001,"5min":10.403,"15min":10.403},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.81699999999999995,"15min":0.75900000000000001},"min":{"1min":0.24399999999999999,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":4.532,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":3.016}]}]},{"osd":0,"up_from":8,"seq":34359739195,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6731244,"kb_used_data":4806432,"kb_used_omap":592,"kb_used_meta":1924207,"kb_avail":87640596,"statfs":{"total":96636764160,"available":89743970304,"internally_reserved":0,"allocated":4921786368,"data_stored":7930738504,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":1970388578},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:07:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.65800000000000003,"15min":0.65900000000000003},"min":{"1min":0.19900000000000001,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":1.5209999999999999,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.90500000000000003,"15min":1.077},"min":{"1min":0.22500000000000001,"5min":0.214,"15min":0.191},"max":{"1min":1.5409999999999999,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.36499999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:07:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.21,"15min":1.1040000000000001},"min":{"1min":0.222,"5min":0.20799999999999999,"15min":0.185},"max":{"1min":4.5990000000000002,"5min":12.622999999999999,"15min":12.622999999999999},"last":4.0359999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.98599999999999999,"15min":1.0369999999999999},"min":{"1min":0.23899999999999999,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":3.302,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.42899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4921069568,"data_stored":7930107763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8074350592,"data_stored":13012393004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3153567744,"data_stored":5082571961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:08:46.516 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.516 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.517 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.517 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.517 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.518 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.518 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.519 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:08:46.519 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:08:46.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.521+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:46.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.609+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:08:46.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.609+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:08:46.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.609+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:46.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:46.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.645+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:46.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:46.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:46.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:47.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.429+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:47.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:47.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:47.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:47.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:47.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.449+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:47.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.449+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:47.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:47.449+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:49.389 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:49.385+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:08:50.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:50.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:50.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:50.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:50.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.521+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:50.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.529+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:50.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.785+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:50.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:50.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:50.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:50.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:50.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:51.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:51.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:51.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:51.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:51.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:51.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.345+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:51.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.345+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:51.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:51.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:53.149 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:53.145+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:08:53.177 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:53.173+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:53.373 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:53.369+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:08:53.385 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:53.377+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:54.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.233+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:54.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:54.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:54.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.241+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:54.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.241+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:54.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.261+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:54.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:54.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:54.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:54.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:54.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:54.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.393+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:54.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.393+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:54.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:54.405+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:55.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.861+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:08:55.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.861+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:55.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:55.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:08:55.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:08:55.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.905+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:08:55.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.905+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:08:55.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:55.925+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:56.245 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:56.241+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:08:56.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:56.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:56.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:56.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:08:56.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:56.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:56.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:56.321+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:57.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:57.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:08:57.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:57.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:08:57.320 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:57.317+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:08:57.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:08:57.337+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:08:58.279 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:08:58.277+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:00.065 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:00.061+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:01.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:01.933+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:09:01.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:01.985+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:01.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:01.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:02.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:02.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:02.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:02.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.041+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:02.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.041+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:02.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.041+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:02.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.045+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:02.346 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:02.341+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:09:02.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:02.357+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:02.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:02.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:02.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:02.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:02.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.785+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:02.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.785+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:02.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:02.793+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:04.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.517+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:09:04.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.521+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:09:04.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.561+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:04.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:04.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.565+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:04.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:04.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:04.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:04.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.589+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:04.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.589+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:04.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:04.593+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:06.520 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:09:07.319 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:09:07.322 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:09:07.364 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2205,"stamp":"2023-10-14T01:09:06.617146+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27214138956,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2014278,"num_read_kb":7830303,"num_write":1980529,"num_write_kb":7921905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80390,"ondisk_log_size":80390,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21019756,"kb_used_data":15843340,"kb_used_omap":1903,"kb_used_meta":5174480,"kb_avail":262095764,"statfs":{"total":289910292480,"available":268386062336,"internally_reserved":0,"allocated":16223580160,"data_stored":26098861957,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1949374,"internal_metadata":5298667842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,2,2,10,13,11,20,12,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17780736,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4179,"num_read_kb":16656,"num_write":4346,"num_write_kb":17384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001892"},"pg_stats":[{"pgid":"2.7","version":"122'232463","reported_seq":462389,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:01.878431+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:09:01.878431+0000","last_peered":"2023-10-14T01:09:01.878431+0000","last_clean":"2023-10-14T01:09:01.878431+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:09:01.878431+0000","last_undegraded":"2023-10-14T01:09:01.878431+0000","last_fullsized":"2023-10-14T01:09:01.878431+0000","mapping_epoch":13,"log_start":"119'222411","ondisk_log_start":"119'222411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3168211127,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":230793,"num_read_kb":908604,"num_write":231407,"num_write_kb":925613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"122'243918","reported_seq":482128,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:01.865127+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:09:01.865127+0000","last_peered":"2023-10-14T01:09:01.865127+0000","last_clean":"2023-10-14T01:09:01.865127+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:09:01.865127+0000","last_undegraded":"2023-10-14T01:09:01.865127+0000","last_fullsized":"2023-10-14T01:09:01.865127+0000","mapping_epoch":13,"log_start":"119'233908","ondisk_log_start":"119'233908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3192328459,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":240813,"num_read_kb":926097,"num_write":231751,"num_write_kb":926603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"122'249261","reported_seq":495015,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:01.869379+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:09:01.869379+0000","last_peered":"2023-10-14T01:09:01.869379+0000","last_clean":"2023-10-14T01:09:01.869379+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:09:01.869379+0000","last_undegraded":"2023-10-14T01:09:01.869379+0000","last_fullsized":"2023-10-14T01:09:01.869379+0000","mapping_epoch":13,"log_start":"119'239212","ondisk_log_start":"119'239212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3357037099,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":246799,"num_read_kb":967902,"num_write":244400,"num_write_kb":977479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"122'263448","reported_seq":518301,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:01.880776+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:09:01.880776+0000","last_peered":"2023-10-14T01:09:01.880776+0000","last_clean":"2023-10-14T01:09:01.880776+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:09:01.880776+0000","last_undegraded":"2023-10-14T01:09:01.880776+0000","last_fullsized":"2023-10-14T01:09:01.880776+0000","mapping_epoch":13,"log_start":"119'253410","ondisk_log_start":"119'253410","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3448959407,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":256281,"num_read_kb":992455,"num_write":251290,"num_write_kb":1004842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"122'254334","reported_seq":508239,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:05.722169+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:09:05.722169+0000","last_peered":"2023-10-14T01:09:05.722169+0000","last_clean":"2023-10-14T01:09:05.722169+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:09:05.722169+0000","last_undegraded":"2023-10-14T01:09:05.722169+0000","last_fullsized":"2023-10-14T01:09:05.722169+0000","mapping_epoch":13,"log_start":"119'244312","ondisk_log_start":"119'244312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":666,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 214s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3400847831,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":255026,"num_read_kb":980323,"num_write":248341,"num_write_kb":992857,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"122'266435","reported_seq":530037,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:03.521507+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:09:03.521507+0000","last_peered":"2023-10-14T01:09:03.521507+0000","last_clean":"2023-10-14T01:09:03.521507+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:09:03.521507+0000","last_undegraded":"2023-10-14T01:09:03.521507+0000","last_fullsized":"2023-10-14T01:09:03.521507+0000","mapping_epoch":13,"log_start":"119'256411","ondisk_log_start":"119'256411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":595,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 211s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3608785316,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":264604,"num_read_kb":1033256,"num_write":261422,"num_write_kb":1045514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"122'263829","reported_seq":523456,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:03.537289+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:09:03.537289+0000","last_peered":"2023-10-14T01:09:03.537289+0000","last_clean":"2023-10-14T01:09:03.537289+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:09:03.537289+0000","last_undegraded":"2023-10-14T01:09:03.537289+0000","last_fullsized":"2023-10-14T01:09:03.537289+0000","mapping_epoch":13,"log_start":"119'253814","ondisk_log_start":"119'253814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":659,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 209s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3559805281,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":260671,"num_read_kb":1018222,"num_write":258770,"num_write_kb":1034890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"122'268767","reported_seq":526514,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:01.887069+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:09:01.887069+0000","last_peered":"2023-10-14T01:09:01.887069+0000","last_clean":"2023-10-14T01:09:01.887069+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:09:01.887069+0000","last_undegraded":"2023-10-14T01:09:01.887069+0000","last_fullsized":"2023-10-14T01:09:01.887069+0000","mapping_epoch":13,"log_start":"119'258714","ondisk_log_start":"119'258714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3477574068,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":259165,"num_read_kb":1003335,"num_write":252979,"num_write_kb":1011559,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":325,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-10-14T01:07:48.370437+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:07:48.370437+0000","last_peered":"2023-10-14T01:07:48.370437+0000","last_clean":"2023-10-14T01:07:48.370437+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:07:48.370437+0000","last_undegraded":"2023-10-14T01:07:48.370437+0000","last_fullsized":"2023-10-14T01:07:48.370437+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27213548588,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2014152,"num_read_kb":7830194,"num_write":1980360,"num_write_kb":7919357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16220860416,"data_stored":26096945240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80263,"ondisk_log_size":80263,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739199,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4636256,"kb_used_data":3093336,"kb_used_omap":349,"kb_used_meta":1542562,"kb_avail":89735584,"statfs":{"total":96636764160,"available":91889238016,"internally_reserved":0,"allocated":3167576064,"data_stored":5095957256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1579583825},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,7,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":0.95499999999999996,"15min":0.80200000000000005},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.17299999999999999},"max":{"1min":5.1210000000000004,"5min":7.6390000000000002,"15min":16.474},"last":5.1609999999999996},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.0269999999999999,"15min":0.89400000000000002},"min":{"1min":0.217,"5min":0.187,"15min":0.187},"max":{"1min":11.44,"5min":11.44,"15min":15.986000000000001},"last":1.216}]},{"osd":1,"last update":"Sat Oct 14 01:08:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.4970000000000001,"15min":1.74},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.20200000000000001},"max":{"1min":4.5069999999999997,"5min":13.519,"15min":14.303000000000001},"last":5.2069999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.544,"15min":1.798},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":4.5359999999999996,"5min":13.565,"15min":14.35},"last":5.2350000000000003}]}]},{"osd":1,"up_from":8,"seq":34359739191,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9547084,"kb_used_data":7918912,"kb_used_omap":961,"kb_used_meta":1627198,"kb_avail":84824756,"statfs":{"total":96636764160,"available":86860550144,"internally_reserved":0,"allocated":8108965888,"data_stored":13046918299,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1666251151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,2,0,7,6,4,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:08:47 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.22900000000000001,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.76300000000000001},"min":{"1min":0.245,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":4.423,"5min":4.423,"15min":10.161},"last":0.56200000000000006}]},{"osd":2,"last update":"Sat Oct 14 01:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":0.90700000000000003,"15min":0.82099999999999995},"min":{"1min":0.34799999999999998,"5min":0.23899999999999999,"15min":0.20899999999999999},"max":{"1min":8.4749999999999996,"5min":10.403,"15min":10.403},"last":0.59199999999999997},{"interface":"front","average":{"1min":2.0099999999999998,"5min":1.1439999999999999,"15min":0.86699999999999999},"min":{"1min":0.254,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":7.2240000000000002,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":0.24099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739200,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6836416,"kb_used_data":4831092,"kb_used_omap":592,"kb_used_meta":2004719,"kb_avail":87535424,"statfs":{"total":96636764160,"available":89636274176,"internally_reserved":0,"allocated":4947038208,"data_stored":7955986402,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":606622,"internal_metadata":2052832866},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.53600000000000003,"15min":0.64700000000000002},"min":{"1min":0.24399999999999999,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":0.59599999999999997,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.73099999999999998,"15min":0.98199999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.191},"max":{"1min":0.73799999999999999,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.311}]},{"osd":2,"last update":"Sat Oct 14 01:08:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1140000000000001,"15min":1.046},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.185},"max":{"1min":5.1079999999999997,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.80500000000000005,"15min":0.95299999999999996},"min":{"1min":0.22800000000000001,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":2.7229999999999999,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.34200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4946419712,"data_stored":7955457907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8107892736,"data_stored":13045935148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3166547968,"data_stored":5095552185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:09:07.365 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.365 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.366 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.366 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.367 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.367 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.367 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.368 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:07.368 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:09:07.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:07.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:07.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:07.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:07.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:07.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:07.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:07.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:07.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:07.737+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:07.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:07.769+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:10.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:10.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:10.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:10.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:10.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.149+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:10.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.177+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:10.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.309+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:10.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:10.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:10.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:10.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:10.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.353+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:10.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.353+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:10.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.361+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:10.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:10.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:10.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:10.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:10.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:10.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:10.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.509+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:10.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.509+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:10.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:10.509+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:11.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:11.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:11.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:11.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.641+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:11.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.941+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:09:11.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.941+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:09:11.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.941+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:11.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:11.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:11.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:11.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:11.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:11.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.969+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:11.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.969+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:11.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:11.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:12.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.561+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:12.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.561+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:12.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:12.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.573+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:12.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:12.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:12.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.613+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:12.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.613+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:12.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:12.613+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:13.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.793+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:13.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:13.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:13.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:13.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:13.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:13.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:13.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:13.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:13.825+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:13.895 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:13.893+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:15.365 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:15.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:15.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:15.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:15.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:15.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.397+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:15.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.397+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:15.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:15.397+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:15.415 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:15.413+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:09:15.430 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:15.429+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:17.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:17.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:17.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:17.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:17.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:17.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:17.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:17.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:17.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:17.064+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:17.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:17.064+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:17.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:17.068+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:18.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:18.504+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:21.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.496+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:21.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:21.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:21.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:21.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:21.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.972+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:21.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.972+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:21.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:21.972+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:22.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:22.860+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:22.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:22.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:22.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:22.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:22.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:22.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:22.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:22.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:22.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:22.876+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:22.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:22.876+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:23.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.288+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:23.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:23.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:23.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:23.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:23.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.420+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:23.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.420+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:23.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:23.420+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:25.147 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:25.140+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:26.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.380+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:26.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:26.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:26.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:26.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:26.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:26.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.440+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:26.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.440+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:26.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:26.456+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:27.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:27.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:27.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:27.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:27.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:27.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:27.369 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:09:27.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:27.412+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:27.609 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:27.604+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:09:27.627 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:27.624+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:28.233 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:09:28.234 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:09:28.288 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2215,"stamp":"2023-10-14T01:09:26.624229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27248102988,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2022522,"num_read_kb":7863183,"num_write":1988807,"num_write_kb":7955017,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80575,"ondisk_log_size":80575,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21151172,"kb_used_data":15908640,"kb_used_omap":1905,"kb_used_meta":5240590,"kb_avail":261964348,"statfs":{"total":289910292480,"available":268251492352,"internally_reserved":0,"allocated":16290447360,"data_stored":26165729157,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1951054,"internal_metadata":5366364850},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,6,11,8,13,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17960960,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4554,"num_read_kb":18164,"num_write":4374,"num_write_kb":17496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005743"},"pg_stats":[{"pgid":"2.7","version":"122'233473","reported_seq":464364,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:22.468712+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:09:22.468712+0000","last_peered":"2023-10-14T01:09:22.468712+0000","last_clean":"2023-10-14T01:09:22.468712+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:09:22.468712+0000","last_undegraded":"2023-10-14T01:09:22.468712+0000","last_fullsized":"2023-10-14T01:09:22.468712+0000","mapping_epoch":13,"log_start":"119'223411","ondisk_log_start":"119'223411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3172348087,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":231759,"num_read_kb":912460,"num_write":232417,"num_write_kb":929653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"122'244996","reported_seq":484195,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:22.416067+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:09:22.416067+0000","last_peered":"2023-10-14T01:09:22.416067+0000","last_clean":"2023-10-14T01:09:22.416067+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:09:22.416067+0000","last_undegraded":"2023-10-14T01:09:22.416067+0000","last_fullsized":"2023-10-14T01:09:22.416067+0000","mapping_epoch":13,"log_start":"119'234908","ondisk_log_start":"119'234908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3196563723,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":241790,"num_read_kb":929993,"num_write":232784,"num_write_kb":930735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"122'250314","reported_seq":497088,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:22.428915+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:09:22.428915+0000","last_peered":"2023-10-14T01:09:22.428915+0000","last_clean":"2023-10-14T01:09:22.428915+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:09:22.428915+0000","last_undegraded":"2023-10-14T01:09:22.428915+0000","last_fullsized":"2023-10-14T01:09:22.428915+0000","mapping_epoch":13,"log_start":"119'240312","ondisk_log_start":"119'240312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3361313323,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":247816,"num_read_kb":971970,"num_write":245438,"num_write_kb":981631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"122'264563","reported_seq":520465,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:22.424267+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:09:22.424267+0000","last_peered":"2023-10-14T01:09:22.424267+0000","last_clean":"2023-10-14T01:09:22.424267+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:09:22.424267+0000","last_undegraded":"2023-10-14T01:09:22.424267+0000","last_fullsized":"2023-10-14T01:09:22.424267+0000","mapping_epoch":13,"log_start":"119'254510","ondisk_log_start":"119'254510","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3453346223,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":257319,"num_read_kb":996595,"num_write":252360,"num_write_kb":1009122,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"122'255344","reported_seq":510270,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:25.903028+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:09:25.903028+0000","last_peered":"2023-10-14T01:09:25.903028+0000","last_clean":"2023-10-14T01:09:25.903028+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:09:25.903028+0000","last_undegraded":"2023-10-14T01:09:25.903028+0000","last_fullsized":"2023-10-14T01:09:25.903028+0000","mapping_epoch":13,"log_start":"119'245312","ondisk_log_start":"119'245312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":723,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 234s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3404796375,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":256020,"num_read_kb":984291,"num_write":249302,"num_write_kb":996701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"122'267583","reported_seq":532295,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:23.885369+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:09:23.885369+0000","last_peered":"2023-10-14T01:09:23.885369+0000","last_clean":"2023-10-14T01:09:23.885369+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:09:23.885369+0000","last_undegraded":"2023-10-14T01:09:23.885369+0000","last_fullsized":"2023-10-14T01:09:23.885369+0000","mapping_epoch":13,"log_start":"119'257511","ondisk_log_start":"119'257511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":639,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 231s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3613327780,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":265697,"num_read_kb":1037608,"num_write":262531,"num_write_kb":1049950,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"122'264958","reported_seq":525655,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:23.925146+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:09:23.925146+0000","last_peered":"2023-10-14T01:09:23.925146+0000","last_clean":"2023-10-14T01:09:23.925146+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:09:23.925146+0000","last_undegraded":"2023-10-14T01:09:23.925146+0000","last_fullsized":"2023-10-14T01:09:23.925146+0000","mapping_epoch":13,"log_start":"119'254914","ondisk_log_start":"119'254914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":728,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 229s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3564224865,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":261727,"num_read_kb":1022434,"num_write":259845,"num_write_kb":1039190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"122'269809","reported_seq":528672,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:22.418690+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:09:22.418690+0000","last_peered":"2023-10-14T01:09:22.418690+0000","last_clean":"2023-10-14T01:09:22.418690+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:09:22.418690+0000","last_undegraded":"2023-10-14T01:09:22.418690+0000","last_fullsized":"2023-10-14T01:09:22.418690+0000","mapping_epoch":13,"log_start":"119'259714","ondisk_log_start":"119'259714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3481592244,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":260268,"num_read_kb":1007723,"num_write":253961,"num_write_kb":1015487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":326,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:08:48.387483+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:08:48.387483+0000","last_peered":"2023-10-14T01:08:48.387483+0000","last_clean":"2023-10-14T01:08:48.387483+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:08:48.387483+0000","last_undegraded":"2023-10-14T01:08:48.387483+0000","last_fullsized":"2023-10-14T01:08:48.387483+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27247512620,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2022396,"num_read_kb":7863074,"num_write":1988638,"num_write_kb":7952469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16288444416,"data_stored":26164529240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80448,"ondisk_log_size":80448,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739203,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4698504,"kb_used_data":3105660,"kb_used_omap":349,"kb_used_meta":1592482,"kb_avail":89673336,"statfs":{"total":96636764160,"available":91825496064,"internally_reserved":0,"allocated":3180195840,"data_stored":5108577032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1630701905},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,2,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.82699999999999996,"15min":0.80100000000000005},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.1609999999999996,"5min":6.0880000000000001,"15min":16.474},"last":0.36099999999999999},{"interface":"front","average":{"1min":1.3129999999999999,"5min":0.93300000000000005,"15min":0.877},"min":{"1min":0.217,"5min":0.187,"15min":0.187},"max":{"1min":11.554,"5min":11.554,"15min":13.326000000000001},"last":0.41999999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.5640000000000001,"15min":1.714},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":6.6799999999999997,"5min":13.519,"15min":14.303000000000001},"last":2.242},{"interface":"front","average":{"1min":2.3180000000000001,"5min":1.726,"15min":1.7989999999999999},"min":{"1min":0.29399999999999998,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":11.593999999999999,"5min":13.565,"15min":14.35},"last":2.2749999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739195,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9691468,"kb_used_data":7951616,"kb_used_omap":961,"kb_used_meta":1738878,"kb_avail":84680372,"statfs":{"total":96636764160,"available":86712700928,"internally_reserved":0,"allocated":8142454784,"data_stored":13080407195,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1780611471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,4,6,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:08:47 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.22900000000000001,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.76300000000000001},"min":{"1min":0.245,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":4.423,"5min":4.423,"15min":10.161},"last":0.66600000000000004}]},{"osd":2,"last update":"Sat Oct 14 01:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":0.90700000000000003,"15min":0.82099999999999995},"min":{"1min":0.34799999999999998,"5min":0.23899999999999999,"15min":0.20899999999999999},"max":{"1min":8.4749999999999996,"5min":10.403,"15min":10.403},"last":0.33500000000000002},{"interface":"front","average":{"1min":2.0099999999999998,"5min":1.1439999999999999,"15min":0.86699999999999999},"min":{"1min":0.254,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":7.2240000000000002,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":0.59899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359739204,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6761200,"kb_used_data":4851364,"kb_used_omap":594,"kb_used_meta":1909229,"kb_avail":87610640,"statfs":{"total":96636764160,"available":89713295360,"internally_reserved":0,"allocated":4967796736,"data_stored":7976744930,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1955051474},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.53600000000000003,"15min":0.64700000000000002},"min":{"1min":0.24399999999999999,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":0.59599999999999997,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.73099999999999998,"15min":0.98199999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.191},"max":{"1min":0.73799999999999999,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.371}]},{"osd":2,"last update":"Sat Oct 14 01:08:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1140000000000001,"15min":1.046},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.185},"max":{"1min":5.1079999999999997,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.80500000000000005,"15min":0.95299999999999996},"min":{"1min":0.22800000000000001,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":2.7229999999999999,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.30099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4967088128,"data_stored":7976126323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8141955072,"data_stored":13079997484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3179401216,"data_stored":5108405433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:09:28.289 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.289 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.290 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.290 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.291 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.291 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.291 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.292 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:28.292 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:09:30.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.416+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:30.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:30.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:30.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:30.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:30.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.488+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:30.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.488+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:30.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:30.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:32.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.248+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:09:32.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.248+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:32.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:32.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:32.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:32.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:32.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:32.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:32.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:32.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.288+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:32.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:32.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:32.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:32.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:32.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:32.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.516+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:32.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.516+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:32.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:32.528+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:32.795 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:32.792+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:32.928 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:32.924+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:33.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.092+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:09:33.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.092+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:09:33.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.124+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:33.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:33.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:33.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:33.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:33.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:33.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.232+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:33.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.232+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:33.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.236+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:33.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.248+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:09:33.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.248+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:09:33.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.248+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:33.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:33.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:33.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:33.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:33.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:33.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.252+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:33.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.252+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:33.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.252+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:33.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:33.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:33.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:33.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:33.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:33.452+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:34.390 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:34.384+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:34.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:34.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:34.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:34.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:36.527 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:36.524+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:38.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:38.472+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:38.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:38.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:38.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:38.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:38.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:38.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:38.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:38.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:38.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:38.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:39.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:39.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:39.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:39.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:39.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:39.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:39.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:39.492+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:40.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:40.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:40.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:40.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:40.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:40.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.900+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:40.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.900+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:40.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:40.916+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:41.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:41.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:41.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:41.616+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:41.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:41.680+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:42.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.236+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:42.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:42.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:42.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:42.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:42.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:42.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.264+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:42.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.264+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:42.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.308+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:42.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:42.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:42.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:42.418 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:42.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:42.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.428+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:42.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.428+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:42.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:42.428+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:43.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:43.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:43.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:43.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:43.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.212+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:43.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.212+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:43.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.256+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:43.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:43.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:43.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:43.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:44.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:44.012+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:46.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:46.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:46.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:46.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:46.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:46.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.188+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:46.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.208+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:46.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:46.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:46.703 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:46.704+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:46.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:46.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:46.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:46.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.740+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:46.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:46.744+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:46.942 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:46.940+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:47.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.608+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:47.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:47.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:47.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:47.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:47.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:47.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.660+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:47.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.660+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:47.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.660+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:47.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.816+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:47.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:47.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:47.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:47.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:47.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:47.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:47.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:47.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:47.844+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:48.293 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:09:48.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.884+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:48.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:48.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.900+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:48.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:48.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:48.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:48.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:48.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:48.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:48.908+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:49.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.128+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:49.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:49.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:49.170 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:09:49.170 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:09:49.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:49.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.172+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:49.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.172+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:49.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.172+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:49.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:49.172+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:49.224 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2226,"stamp":"2023-10-14T01:09:48.631701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27286875724,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2032154,"num_read_kb":7901607,"num_write":1998246,"num_write_kb":7992773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80661,"ondisk_log_size":80661,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21497948,"kb_used_data":15981764,"kb_used_omap":1905,"kb_used_meta":5514254,"kb_avail":261617572,"statfs":{"total":289910292480,"available":267896393728,"internally_reserved":0,"allocated":16365326336,"data_stored":26240608133,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1951054,"internal_metadata":5646596786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,1,6,6,7,18,20,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21680128,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5395,"num_read_kb":21528,"num_write":5264,"num_write_kb":21056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002025"},"pg_stats":[{"pgid":"2.7","version":"122'234721","reported_seq":466778,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:48.317533+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:09:48.317533+0000","last_peered":"2023-10-14T01:09:48.317533+0000","last_clean":"2023-10-14T01:09:48.317533+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:09:48.317533+0000","last_undegraded":"2023-10-14T01:09:48.317533+0000","last_fullsized":"2023-10-14T01:09:48.317533+0000","mapping_epoch":13,"log_start":"119'224711","ondisk_log_start":"119'224711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3177451703,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":232923,"num_read_kb":917108,"num_write":233665,"num_write_kb":934645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"122'246308","reported_seq":486760,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:48.299045+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:09:48.299045+0000","last_peered":"2023-10-14T01:09:48.299045+0000","last_clean":"2023-10-14T01:09:48.299045+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:09:48.299045+0000","last_undegraded":"2023-10-14T01:09:48.299045+0000","last_fullsized":"2023-10-14T01:09:48.299045+0000","mapping_epoch":13,"log_start":"119'236208","ondisk_log_start":"119'236208","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3201667339,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":243031,"num_read_kb":934945,"num_write":234020,"num_write_kb":935679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"122'251553","reported_seq":499646,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:48.288359+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:09:48.288359+0000","last_peered":"2023-10-14T01:09:48.288359+0000","last_clean":"2023-10-14T01:09:48.288359+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:09:48.288359+0000","last_undegraded":"2023-10-14T01:09:48.288359+0000","last_fullsized":"2023-10-14T01:09:48.288359+0000","mapping_epoch":13,"log_start":"119'241512","ondisk_log_start":"119'241512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3366302251,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":249129,"num_read_kb":977210,"num_write":246651,"num_write_kb":986483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"122'266004","reported_seq":523250,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:48.277251+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:09:48.277251+0000","last_peered":"2023-10-14T01:09:48.277251+0000","last_clean":"2023-10-14T01:09:48.277251+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:09:48.277251+0000","last_undegraded":"2023-10-14T01:09:48.277251+0000","last_fullsized":"2023-10-14T01:09:48.277251+0000","mapping_epoch":13,"log_start":"120'255910","ondisk_log_start":"120'255910","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3458994607,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":258648,"num_read_kb":1001891,"num_write":253726,"num_write_kb":1014586,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"122'256372","reported_seq":512338,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:46.022007+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:09:46.022007+0000","last_peered":"2023-10-14T01:09:46.022007+0000","last_clean":"2023-10-14T01:09:46.022007+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:09:46.022007+0000","last_undegraded":"2023-10-14T01:09:46.022007+0000","last_fullsized":"2023-10-14T01:09:46.022007+0000","mapping_epoch":13,"log_start":"119'246312","ondisk_log_start":"119'246312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":780,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 255s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3408785879,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":257032,"num_read_kb":988331,"num_write":250278,"num_write_kb":1000605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"122'268685","reported_seq":534523,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:44.215731+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:09:44.215731+0000","last_peered":"2023-10-14T01:09:44.215731+0000","last_clean":"2023-10-14T01:09:44.215731+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:09:44.215731+0000","last_undegraded":"2023-10-14T01:09:44.215731+0000","last_fullsized":"2023-10-14T01:09:44.215731+0000","mapping_epoch":13,"log_start":"120'258611","ondisk_log_start":"120'258611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":713,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 252s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3617575332,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":266805,"num_read_kb":1042032,"num_write":263567,"num_write_kb":1054094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"122'266086","reported_seq":527918,"reported_epoch":122,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:09:44.192843+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:09:44.192843+0000","last_peered":"2023-10-14T01:09:44.192843+0000","last_clean":"2023-10-14T01:09:44.192843+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:09:44.192843+0000","last_undegraded":"2023-10-14T01:09:44.192843+0000","last_fullsized":"2023-10-14T01:09:44.192843+0000","mapping_epoch":13,"log_start":"120'256014","ondisk_log_start":"120'256014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":779,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 250s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3568656737,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":262850,"num_read_kb":1026902,"num_write":260922,"num_write_kb":1043498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"122'271197","reported_seq":531426,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:48.278757+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:09:48.278757+0000","last_peered":"2023-10-14T01:09:48.278757+0000","last_clean":"2023-10-14T01:09:48.278757+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:09:48.278757+0000","last_undegraded":"2023-10-14T01:09:48.278757+0000","last_fullsized":"2023-10-14T01:09:48.278757+0000","mapping_epoch":13,"log_start":"120'261114","ondisk_log_start":"120'261114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3486851508,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":261610,"num_read_kb":1013079,"num_write":255248,"num_write_kb":1020635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":326,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:08:48.387483+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:08:48.387483+0000","last_peered":"2023-10-14T01:08:48.387483+0000","last_clean":"2023-10-14T01:08:48.387483+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:08:48.387483+0000","last_undegraded":"2023-10-14T01:08:48.387483+0000","last_fullsized":"2023-10-14T01:08:48.387483+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27286285356,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2032028,"num_read_kb":7901498,"num_write":1998077,"num_write_kb":7990225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16363577344,"data_stored":26239662168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80534,"ondisk_log_size":80534,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739207,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4742912,"kb_used_data":3118136,"kb_used_omap":349,"kb_used_meta":1624418,"kb_avail":89628928,"statfs":{"total":96636764160,"available":91780022272,"internally_reserved":0,"allocated":3192971264,"data_stored":5121352456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1663404369},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,2,5,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.82699999999999996,"15min":0.80100000000000005},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.1609999999999996,"5min":6.0880000000000001,"15min":16.474},"last":0.25600000000000001},{"interface":"front","average":{"1min":1.3129999999999999,"5min":0.93300000000000005,"15min":0.877},"min":{"1min":0.217,"5min":0.187,"15min":0.187},"max":{"1min":11.554,"5min":11.554,"15min":13.326000000000001},"last":0.22700000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.5640000000000001,"15min":1.714},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":6.6799999999999997,"5min":13.519,"15min":14.303000000000001},"last":5.665},{"interface":"front","average":{"1min":2.3180000000000001,"5min":1.726,"15min":1.7989999999999999},"min":{"1min":0.29399999999999998,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":11.593999999999999,"5min":13.565,"15min":14.35},"last":5.6920000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739201,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9918784,"kb_used_data":7992760,"kb_used_omap":961,"kb_used_meta":1925054,"kb_avail":84453056,"statfs":{"total":96636764160,"available":86479929344,"internally_reserved":0,"allocated":8184586240,"data_stored":13122538651,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":1971255695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,0,1,4,3,12,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:08:47 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":0.95299999999999996,"15min":0.97699999999999998},"min":{"1min":0.22900000000000001,"5min":0.184,"15min":0.17199999999999999},"max":{"1min":14.957000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.59499999999999997,"15min":0.76300000000000001},"min":{"1min":0.245,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":4.423,"5min":4.423,"15min":10.161},"last":0.42199999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:08:46 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":0.90700000000000003,"15min":0.82099999999999995},"min":{"1min":0.34799999999999998,"5min":0.23899999999999999,"15min":0.20899999999999999},"max":{"1min":8.4749999999999996,"5min":10.403,"15min":10.403},"last":0.35299999999999998},{"interface":"front","average":{"1min":2.0099999999999998,"5min":1.1439999999999999,"15min":0.86699999999999999},"min":{"1min":0.254,"5min":0.20100000000000001,"15min":0.193},"max":{"1min":7.2240000000000002,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":0.44500000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739208,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6836252,"kb_used_data":4870868,"kb_used_omap":594,"kb_used_meta":1964781,"kb_avail":87535588,"statfs":{"total":96636764160,"available":89636442112,"internally_reserved":0,"allocated":4987768832,"data_stored":7996717026,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":2011936722},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1,3,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:08:51 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.53600000000000003,"15min":0.64700000000000002},"min":{"1min":0.24399999999999999,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":0.59599999999999997,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.314},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.73099999999999998,"15min":0.98199999999999998},"min":{"1min":0.22700000000000001,"5min":0.214,"15min":0.191},"max":{"1min":0.73799999999999999,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.29499999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:08:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.1140000000000001,"15min":1.046},"min":{"1min":0.26200000000000001,"5min":0.20799999999999999,"15min":0.185},"max":{"1min":5.1079999999999997,"5min":12.622999999999999,"15min":12.622999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.80500000000000005,"15min":0.95299999999999996},"min":{"1min":0.22800000000000001,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":2.7229999999999999,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.25800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4987179008,"data_stored":7996217203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8184160256,"data_stored":13122202668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3192238080,"data_stored":5121242297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:09:49.225 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.226 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.226 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.227 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.227 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.227 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.228 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.228 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:09:49.228 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:09:50.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:50.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:50.220 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:50.220+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:50.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:50.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:50.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:50.232+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:50.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:50.232+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:50.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:50.232+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:50.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:50.252+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:52.080 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:52.080+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:09:52.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:52.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:53.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:53.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:53.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:53.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.032+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:53.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.032+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:53.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.504+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:53.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:53.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:53.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:53.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:53.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:53.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.532+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:53.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.532+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:53.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:53.532+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:54.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:54.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.799+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:54.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:54.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:54.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:54.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:54.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.827+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:54.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:54.863+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:55.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.383+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:55.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:55.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:55.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:55.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:55.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:55.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.435+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:55.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.435+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:09:55.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:55.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:56.055 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:09:56.055+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:09:56.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:56.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:56.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:56.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:57.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:57.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:09:57.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.015+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:09:57.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.051+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:09:57.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.807+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:09:57.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.823+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:09:57.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:09:57.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.827+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:09:57.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.839+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:09:57.841 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:09:57.839+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:01.285 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:10:01.283+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:10:01.286 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:10:01.283+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:10:01.286 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:10:01.283+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T01:10:01.286 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:10:01.283+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aff790 2023-10-14T01:10:01.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.435+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:01.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:01.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:01.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:01.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:01.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:01.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:01.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.463+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:01.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:01.463+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:02.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.119+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:02.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:02.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:02.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:02.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:02.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:02.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.151+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:02.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.151+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:02.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.155+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:02.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:02.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:02.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:02.307+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:03.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.027+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:03.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:03.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:03.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:03.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:03.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:03.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.719+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:03.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.719+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:03.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.751+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:03.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:03.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:03.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:06.170 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:06.171+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:10:08.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.751+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:08.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:08.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:08.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:08.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:08.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:08.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.819+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:08.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.819+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:08.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:08.827+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:09.230 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:10:10.053 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:10:10.053 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:10:10.104 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2237,"stamp":"2023-10-14T01:10:08.642867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27315203660,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2039149,"num_read_kb":7929507,"num_write":2005137,"num_write_kb":8020337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80512,"ondisk_log_size":80512,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21768664,"kb_used_data":16037496,"kb_used_omap":1905,"kb_used_meta":5729230,"kb_avail":261346856,"statfs":{"total":289910292480,"available":267619180544,"internally_reserved":0,"allocated":16422395904,"data_stored":26297669731,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":1951054,"internal_metadata":5866732210},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,11,11,21,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":11366400,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2774,"num_read_kb":11068,"num_write":2773,"num_write_kb":11092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.006455"},"pg_stats":[{"pgid":"2.7","version":"123'235476","reported_seq":468266,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-14T01:10:03.716482+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:10:03.716482+0000","last_peered":"2023-10-14T01:10:03.716482+0000","last_clean":"2023-10-14T01:10:03.716482+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:10:03.716482+0000","last_undegraded":"2023-10-14T01:10:03.716482+0000","last_fullsized":"2023-10-14T01:10:03.716482+0000","mapping_epoch":13,"log_start":"119'225411","ondisk_log_start":"119'225411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3180556471,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":233655,"num_read_kb":920032,"num_write":234420,"num_write_kb":937665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"123'247091","reported_seq":488273,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-14T01:10:03.742315+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:10:03.742315+0000","last_peered":"2023-10-14T01:10:03.742315+0000","last_clean":"2023-10-14T01:10:03.742315+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:10:03.742315+0000","last_undegraded":"2023-10-14T01:10:03.742315+0000","last_fullsized":"2023-10-14T01:10:03.742315+0000","mapping_epoch":13,"log_start":"120'237008","ondisk_log_start":"120'237008","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3204714763,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":243749,"num_read_kb":937813,"num_write":234758,"num_write_kb":938631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"123'252334","reported_seq":501211,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-14T01:10:03.717789+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:10:03.717789+0000","last_peered":"2023-10-14T01:10:03.717789+0000","last_clean":"2023-10-14T01:10:03.717789+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:10:03.717789+0000","last_undegraded":"2023-10-14T01:10:03.717789+0000","last_fullsized":"2023-10-14T01:10:03.717789+0000","mapping_epoch":13,"log_start":"120'242312","ondisk_log_start":"120'242312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3369488939,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":249909,"num_read_kb":980310,"num_write":247416,"num_write_kb":989543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"123'266811","reported_seq":524852,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-14T01:10:03.731357+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:10:03.731357+0000","last_peered":"2023-10-14T01:10:03.731357+0000","last_clean":"2023-10-14T01:10:03.731357+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:10:03.731357+0000","last_undegraded":"2023-10-14T01:10:03.731357+0000","last_fullsized":"2023-10-14T01:10:03.731357+0000","mapping_epoch":13,"log_start":"120'256810","ondisk_log_start":"120'256810","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3462111663,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":259432,"num_read_kb":1005019,"num_write":254487,"num_write_kb":1017630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"123'257381","reported_seq":514406,"reported_epoch":123,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:06.169886+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:10:06.169886+0000","last_peered":"2023-10-14T01:10:06.169886+0000","last_clean":"2023-10-14T01:10:06.169886+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:10:06.169886+0000","last_undegraded":"2023-10-14T01:10:06.169886+0000","last_fullsized":"2023-10-14T01:10:06.169886+0000","mapping_epoch":13,"log_start":"120'247312","ondisk_log_start":"120'247312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":857,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 275s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3412660695,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":258061,"num_read_kb":992431,"num_write":251218,"num_write_kb":1004365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"123'269810","reported_seq":536767,"reported_epoch":123,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:04.380093+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:10:04.380093+0000","last_peered":"2023-10-14T01:10:04.380093+0000","last_clean":"2023-10-14T01:10:04.380093+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:10:04.380093+0000","last_undegraded":"2023-10-14T01:10:04.380093+0000","last_fullsized":"2023-10-14T01:10:04.380093+0000","mapping_epoch":13,"log_start":"120'259711","ondisk_log_start":"120'259711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":774,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 272s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3621917092,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":267905,"num_read_kb":1046420,"num_write":264630,"num_write_kb":1058346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"123'267216","reported_seq":530123,"reported_epoch":123,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:04.378192+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:10:04.378192+0000","last_peered":"2023-10-14T01:10:04.378192+0000","last_clean":"2023-10-14T01:10:04.378192+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:10:04.378192+0000","last_undegraded":"2023-10-14T01:10:04.378192+0000","last_fullsized":"2023-10-14T01:10:04.378192+0000","mapping_epoch":13,"log_start":"120'257214","ondisk_log_start":"120'257214","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":833,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 270s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3573039457,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":263904,"num_read_kb":1031106,"num_write":261990,"num_write_kb":1047770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"123'272058","reported_seq":533096,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-14T01:10:03.713584+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:10:03.713584+0000","last_peered":"2023-10-14T01:10:03.713584+0000","last_clean":"2023-10-14T01:10:03.713584+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:10:03.713584+0000","last_undegraded":"2023-10-14T01:10:03.713584+0000","last_fullsized":"2023-10-14T01:10:03.713584+0000","mapping_epoch":13,"log_start":"120'262014","ondisk_log_start":"120'262014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3490124212,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":262408,"num_read_kb":1016267,"num_write":256049,"num_write_kb":1023839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":327,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-10-14T01:09:12.192218+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:09:12.192218+0000","last_peered":"2023-10-14T01:09:12.192218+0000","last_clean":"2023-10-14T01:09:12.192218+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:09:12.192218+0000","last_undegraded":"2023-10-14T01:09:12.192218+0000","last_fullsized":"2023-10-14T01:09:12.192218+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27314613292,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2039023,"num_read_kb":7929398,"num_write":2004968,"num_write_kb":8017789,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16421638144,"data_stored":26297722968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80385,"ondisk_log_size":80385,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739211,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4805544,"kb_used_data":3130844,"kb_used_omap":349,"kb_used_meta":1674338,"kb_avail":89566296,"statfs":{"total":96636764160,"available":91715887104,"internally_reserved":0,"allocated":3205984256,"data_stored":5134361463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1714522449},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,5,2,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.82699999999999996,"15min":0.80100000000000005},"min":{"1min":0.17699999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.1609999999999996,"5min":6.0880000000000001,"15min":16.474},"last":0.27800000000000002},{"interface":"front","average":{"1min":1.3129999999999999,"5min":0.93300000000000005,"15min":0.877},"min":{"1min":0.217,"5min":0.187,"15min":0.187},"max":{"1min":11.554,"5min":11.554,"15min":13.326000000000001},"last":0.23400000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:09:07 2023","interfaces":[{"interface":"back","average":{"1min":1.7070000000000001,"5min":1.5640000000000001,"15min":1.714},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":6.6799999999999997,"5min":13.519,"15min":14.303000000000001},"last":4.0060000000000002},{"interface":"front","average":{"1min":2.3180000000000001,"5min":1.726,"15min":1.7989999999999999},"min":{"1min":0.29399999999999998,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":11.593999999999999,"5min":13.565,"15min":14.35},"last":4.0389999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739204,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10032824,"kb_used_data":8016624,"kb_used_omap":961,"kb_used_meta":2015230,"kb_avail":84339016,"statfs":{"total":96636764160,"available":86363152384,"internally_reserved":0,"allocated":8209022976,"data_stored":13146975387,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":984689,"internal_metadata":2063595919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,7,8,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:09:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.90700000000000003,"15min":0.95299999999999996},"min":{"1min":0.19700000000000001,"5min":0.192,"15min":0.17199999999999999},"max":{"1min":3.8650000000000002,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.625,"15min":0.73499999999999999},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":3.911,"5min":4.423,"15min":10.161},"last":0.442}]},{"osd":2,"last update":"Sat Oct 14 01:09:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.89400000000000002,"15min":0.78700000000000003},"min":{"1min":0.25600000000000001,"5min":0.23899999999999999,"15min":0.20899999999999999},"max":{"1min":1.242,"5min":10.403,"15min":10.403},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":1.038,"15min":0.872},"min":{"1min":0.221,"5min":0.20100000000000001,"15min":0.19500000000000001},"max":{"1min":2.7639999999999998,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":0.34899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359739212,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6930296,"kb_used_data":4890028,"kb_used_omap":594,"kb_used_meta":2039661,"kb_avail":87441544,"statfs":{"total":96636764160,"available":89540141056,"internally_reserved":0,"allocated":5007388672,"data_stored":8016332881,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":2088613842},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.49099999999999999,"15min":0.61099999999999999},"min":{"1min":0.223,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":0.58499999999999996,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":1.4139999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.61899999999999999,"15min":0.91400000000000003},"min":{"1min":0.28899999999999998,"5min":0.214,"15min":0.191},"max":{"1min":0.63100000000000001,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":1.4570000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":1.0469999999999999},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.185},"max":{"1min":5.3979999999999997,"5min":6.5030000000000001,"15min":12.622999999999999},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.69399999999999995,"15min":0.91000000000000003},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.20799999999999999},"max":{"1min":0.65900000000000003,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":1.4870000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5006958592,"data_stored":8015996787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8209416192,"data_stored":13147458604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3205263360,"data_stored":5134267577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:10:10.105 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.106 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.106 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.107 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.107 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.107 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.108 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.108 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:10.109 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:10:10.855 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:10:10.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:10:11.596 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:10:11.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:10:12.502 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:10:12.541 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:10:13.323 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:10:13.367 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-14T01:10:14.198 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-14T01:10:14.262 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.1 2023-10-14T01:10:15.118 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.1 on osd.2 to deep-scrub 2023-10-14T01:10:15.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.143+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:15.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:15.173 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.0 2023-10-14T01:10:15.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:15.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:15.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:15.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:15.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.227+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:15.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.227+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:15.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:15.227+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:15.817 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.0 on osd.2 to deep-scrub 2023-10-14T01:10:15.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:10:16.899 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:10:16.951 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:10:18.125 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:18.123+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:10:19.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.083+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:19.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:19.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:19.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:19.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:19.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.111+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:19.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.111+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:19.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:19.119+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:20.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.051+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:20.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:20.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:20.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:20.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:20.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:20.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.135+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:20.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.135+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:20.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:20.147+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:24.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:24.983+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:25.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:25.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:25.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:25.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:25.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:25.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.075+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:25.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.075+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:25.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:25.087+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:28.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.115+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:28.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:28.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:28.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:28.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:28.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.175+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:28.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.175+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:28.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.175+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:28.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:28.175+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:29.316 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:29.315+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:10:29.342 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:29.339+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:10:30.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:30.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:30.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:30.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:30.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:30.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.891+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:30.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.891+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:30.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:30.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:31.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:31.159+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:32.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:32.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:32.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:32.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:32.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:32.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:32.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:32.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:32.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:10:32.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:10:32.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:32.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:32.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:32.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:32.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:32.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.327+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:32.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.327+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:32.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.343+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:32.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.351+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:10:32.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.367+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:32.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:32.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:32.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:32.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:32.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:32.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:32.447+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:33.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:33.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:33.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:33.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:33.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:33.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:33.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:33.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:33.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:33.818+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:33.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:33.818+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:33.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:33.834+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:34.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:34.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:35.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:35.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:35.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:35.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:35.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:35.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:35.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:35.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:35.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:35.026+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:35.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:35.062+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:36.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:36.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:36.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:36.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:36.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.402+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:36.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:36.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:36.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:36.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:36.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:36.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.914+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:36.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.914+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:36.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:36.918+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:36.952 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:10:37.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:37.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:37.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:37.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:37.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:37.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:37.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:37.242+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:37.624 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:10:37.625 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:10:37.678 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2252,"stamp":"2023-10-14T01:10:36.647659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27364789836,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2051330,"num_read_kb":7978031,"num_write":2017205,"num_write_kb":8068609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80561,"ondisk_log_size":80561,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21307140,"kb_used_data":16134380,"kb_used_omap":1993,"kb_used_meta":5170742,"kb_avail":261808380,"statfs":{"total":289910292480,"available":268091781120,"internally_reserved":0,"allocated":16521605120,"data_stored":26396862878,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2040936,"internal_metadata":5294840728},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,1,3,6,8,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19759104,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4809,"num_read_kb":19148,"num_write":4829,"num_write_kb":19316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002091"},"pg_stats":[{"pgid":"2.7","version":"124'236905","reported_seq":471142,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:34.354969+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:10:34.354969+0000","last_peered":"2023-10-14T01:10:34.354969+0000","last_clean":"2023-10-14T01:10:34.354969+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:10:34.354969+0000","last_undegraded":"2023-10-14T01:10:34.354969+0000","last_fullsized":"2023-10-14T01:10:34.354969+0000","mapping_epoch":13,"log_start":"120'226811","ondisk_log_start":"120'226811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3186438327,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":235099,"num_read_kb":925788,"num_write":235849,"num_write_kb":943381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"124'248535","reported_seq":491143,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:34.355886+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:10:34.355886+0000","last_peered":"2023-10-14T01:10:34.355886+0000","last_clean":"2023-10-14T01:10:34.355886+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:10:34.355886+0000","last_undegraded":"2023-10-14T01:10:34.355886+0000","last_fullsized":"2023-10-14T01:10:34.355886+0000","mapping_epoch":13,"log_start":"120'238508","ondisk_log_start":"120'238508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3210490123,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":245154,"num_read_kb":943409,"num_write":236166,"num_write_kb":944263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"124'253739","reported_seq":504169,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:34.356619+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:10:34.356619+0000","last_peered":"2023-10-14T01:10:34.356619+0000","last_clean":"2023-10-14T01:10:34.356619+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:10:34.356619+0000","last_undegraded":"2023-10-14T01:10:34.356619+0000","last_fullsized":"2023-10-14T01:10:34.356619+0000","mapping_epoch":13,"log_start":"120'243712","ondisk_log_start":"120'243712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3375206955,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":251454,"num_read_kb":986466,"num_write":248809,"num_write_kb":995115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"124'268431","reported_seq":528008,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:34.357725+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:10:34.357725+0000","last_peered":"2023-10-14T01:10:34.357725+0000","last_clean":"2023-10-14T01:10:34.357725+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:10:34.357725+0000","last_undegraded":"2023-10-14T01:10:34.357725+0000","last_fullsized":"2023-10-14T01:10:34.357725+0000","mapping_epoch":13,"log_start":"120'258410","ondisk_log_start":"120'258410","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3468644783,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":260952,"num_read_kb":1011071,"num_write":256071,"num_write_kb":1023966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"124'258980","reported_seq":517540,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:36.469961+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:10:36.469961+0000","last_peered":"2023-10-14T01:10:36.469961+0000","last_clean":"2023-10-14T01:10:36.469961+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:10:36.469961+0000","last_undegraded":"2023-10-14T01:10:36.469961+0000","last_fullsized":"2023-10-14T01:10:36.469961+0000","mapping_epoch":13,"log_start":"120'248912","ondisk_log_start":"120'248912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":958,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 305s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3418882519,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":259546,"num_read_kb":998347,"num_write":252732,"num_write_kb":1010421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"124'271496","reported_seq":540072,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:34.872132+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:10:34.872132+0000","last_peered":"2023-10-14T01:10:34.872132+0000","last_clean":"2023-10-14T01:10:34.872132+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:10:34.872132+0000","last_undegraded":"2023-10-14T01:10:34.872132+0000","last_fullsized":"2023-10-14T01:10:34.872132+0000","mapping_epoch":13,"log_start":"120'261411","ondisk_log_start":"120'261411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":877,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 302s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3628478884,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":269499,"num_read_kb":1052760,"num_write":266233,"num_write_kb":1064758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"124'268889","reported_seq":533436,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:34.901615+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:10:34.901615+0000","last_peered":"2023-10-14T01:10:34.901615+0000","last_clean":"2023-10-14T01:10:34.901615+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:10:34.901615+0000","last_undegraded":"2023-10-14T01:10:34.901615+0000","last_fullsized":"2023-10-14T01:10:34.901615+0000","mapping_epoch":13,"log_start":"120'258814","ondisk_log_start":"120'258814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":931,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 300s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3579539809,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":265517,"num_read_kb":1037530,"num_write":263583,"num_write_kb":1054142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"124'273651","reported_seq":536293,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:34.318122+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:10:34.318122+0000","last_peered":"2023-10-14T01:10:34.318122+0000","last_clean":"2023-10-14T01:10:34.318122+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:10:34.318122+0000","last_undegraded":"2023-10-14T01:10:34.318122+0000","last_fullsized":"2023-10-14T01:10:34.318122+0000","mapping_epoch":13,"log_start":"120'263614","ondisk_log_start":"120'263614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3496518068,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":263983,"num_read_kb":1022551,"num_write":257593,"num_write_kb":1030015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":328,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-14T01:10:03.433520+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:10:03.433520+0000","last_peered":"2023-10-14T01:10:03.433520+0000","last_clean":"2023-10-14T01:10:03.433520+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:10:03.433520+0000","last_undegraded":"2023-10-14T01:10:03.433520+0000","last_fullsized":"2023-10-14T01:10:03.433520+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27364199468,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2051204,"num_read_kb":7977922,"num_write":2017036,"num_write_kb":8066061,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16520269824,"data_stored":26396354648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80434,"ondisk_log_size":80434,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739217,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4918588,"kb_used_data":3149492,"kb_used_omap":349,"kb_used_meta":1768738,"kb_avail":89453252,"statfs":{"total":96636764160,"available":91600130048,"internally_reserved":0,"allocated":3225079808,"data_stored":5153452987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1811188049},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89300000000000002,"15min":0.78400000000000003},"min":{"1min":0.17799999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.7729999999999997,"5min":6.0880000000000001,"15min":16.474},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.94899999999999995,"15min":0.878},"min":{"1min":0.214,"5min":0.187,"15min":0.187},"max":{"1min":1.891,"5min":11.554,"15min":13.326000000000001},"last":0.44}]},{"osd":1,"last update":"Sat Oct 14 01:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.448,"15min":1.75},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":5.665,"5min":8.3849999999999998,"15min":14.303000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":1.665,"5min":1.611,"15min":1.8360000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":5.6920000000000002,"5min":11.593999999999999,"15min":14.35},"last":0.46100000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739210,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10034508,"kb_used_data":8065860,"kb_used_omap":1049,"kb_used_meta":1967590,"kb_avail":84337332,"statfs":{"total":96636764160,"available":86361427968,"internally_reserved":0,"allocated":8259440640,"data_stored":13197385038,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1074571,"internal_metadata":2014812789},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,1,4,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:09:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.90700000000000003,"15min":0.95299999999999996},"min":{"1min":0.19700000000000001,"5min":0.192,"15min":0.17199999999999999},"max":{"1min":3.8650000000000002,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.38},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.625,"15min":0.73499999999999999},"min":{"1min":0.23699999999999999,"5min":0.222,"15min":0.21299999999999999},"max":{"1min":3.911,"5min":4.423,"15min":10.161},"last":0.41599999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:09:50 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.89400000000000002,"15min":0.78700000000000003},"min":{"1min":0.25600000000000001,"5min":0.23899999999999999,"15min":0.20899999999999999},"max":{"1min":1.242,"5min":10.403,"15min":10.403},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":1.038,"15min":0.872},"min":{"1min":0.221,"5min":0.20100000000000001,"15min":0.19500000000000001},"max":{"1min":2.7639999999999998,"5min":7.9560000000000004,"15min":9.2200000000000006},"last":0.32200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739218,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6354044,"kb_used_data":4919028,"kb_used_omap":594,"kb_used_meta":1434413,"kb_avail":88017796,"statfs":{"total":96636764160,"available":90130223104,"internally_reserved":0,"allocated":5037084672,"data_stored":8046024853,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1468839890},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.49099999999999999,"15min":0.61099999999999999},"min":{"1min":0.223,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":0.58499999999999996,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.61899999999999999,"15min":0.91400000000000003},"min":{"1min":0.28899999999999998,"5min":0.214,"15min":0.191},"max":{"1min":0.63100000000000001,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.35699999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":1.0469999999999999},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.185},"max":{"1min":5.3979999999999997,"5min":6.5030000000000001,"15min":12.622999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.69399999999999995,"15min":0.91000000000000003},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.20799999999999999},"max":{"1min":0.65900000000000003,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.32300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5036761088,"data_stored":8045799283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8258859008,"data_stored":13196901420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3224649728,"data_stored":5153653945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:10:37.680 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.680 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.681 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.681 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.681 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.682 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.682 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.683 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:37.683 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:10:38.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.010+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:10:38.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.010+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:38.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:38.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:38.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:38.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:38.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:38.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.090+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:38.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.090+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:38.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:38.090+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:40.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:40.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:40.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:40.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:40.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:40.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.070+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:40.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.070+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:40.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.070+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:40.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:40.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:40.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:40.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:40.294+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:42.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:42.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:42.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:42.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:42.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:42.762+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:44.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:44.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:44.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:44.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:44.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:44.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:44.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:44.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:44.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:44.834+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:44.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:44.834+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:44.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:44.854+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:45.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:45.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:45.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:45.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:45.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:45.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:45.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:45.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:45.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:45.198+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:45.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:45.198+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:46.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.574+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:46.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:46.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:46.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:46.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:46.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:46.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.674+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:46.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.674+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:46.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.702+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:46.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.710+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:10:46.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.718+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:46.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:46.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:46.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:46.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:46.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:46.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:46.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:46.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:48.912 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:48.902+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:10:50.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.678+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:50.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:50.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:50.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:50.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.766+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:50.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.766+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:50.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:50.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:50.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:50.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:50.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:50.794+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:53.199 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:53.198+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:10:53.214 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:53.210+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:10:53.935 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:10:53.934+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:10:56.932 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.930+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:56.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:56.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:56.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.942+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:56.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:56.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.970+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:56.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.970+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:56.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:56.970+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:10:57.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:10:58.505 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:10:58.506 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:10:58.555 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2262,"stamp":"2023-10-14T01:10:56.652952+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27398258252,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2059665,"num_read_kb":8011207,"num_write":2025379,"num_write_kb":8101305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80457,"ondisk_log_size":80457,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21567888,"kb_used_data":16198964,"kb_used_omap":1993,"kb_used_meta":5366902,"kb_avail":261547632,"statfs":{"total":289910292480,"available":267824775168,"internally_reserved":0,"allocated":16587739136,"data_stored":26462996894,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2040936,"internal_metadata":5495708568},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,4,19,28,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20025344,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5027,"num_read_kb":20008,"num_write":4889,"num_write_kb":19556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003842"},"pg_stats":[{"pgid":"2.7","version":"124'237856","reported_seq":473081,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:54.909368+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:10:54.909368+0000","last_peered":"2023-10-14T01:10:54.909368+0000","last_clean":"2023-10-14T01:10:54.909368+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:10:54.909368+0000","last_undegraded":"2023-10-14T01:10:54.909368+0000","last_fullsized":"2023-10-14T01:10:54.909368+0000","mapping_epoch":13,"log_start":"120'227811","ondisk_log_start":"120'227811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3190329527,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":236087,"num_read_kb":929716,"num_write":236800,"num_write_kb":947185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"124'249587","reported_seq":493160,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:54.910198+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:10:54.910198+0000","last_peered":"2023-10-14T01:10:54.910198+0000","last_clean":"2023-10-14T01:10:54.910198+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:10:54.910198+0000","last_undegraded":"2023-10-14T01:10:54.910198+0000","last_fullsized":"2023-10-14T01:10:54.910198+0000","mapping_epoch":13,"log_start":"120'239508","ondisk_log_start":"120'239508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3214602507,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":246107,"num_read_kb":947209,"num_write":237173,"num_write_kb":948291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"124'254736","reported_seq":506255,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:54.890528+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:10:54.890528+0000","last_peered":"2023-10-14T01:10:54.890528+0000","last_clean":"2023-10-14T01:10:54.890528+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:10:54.890528+0000","last_undegraded":"2023-10-14T01:10:54.890528+0000","last_fullsized":"2023-10-14T01:10:54.890528+0000","mapping_epoch":13,"log_start":"120'244712","ondisk_log_start":"120'244712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3379216939,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":252539,"num_read_kb":990790,"num_write":249791,"num_write_kb":999043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"124'269535","reported_seq":530213,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:54.842409+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:10:54.842409+0000","last_peered":"2023-10-14T01:10:54.842409+0000","last_clean":"2023-10-14T01:10:54.842409+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:10:54.842409+0000","last_undegraded":"2023-10-14T01:10:54.842409+0000","last_fullsized":"2023-10-14T01:10:54.842409+0000","mapping_epoch":13,"log_start":"120'259510","ondisk_log_start":"120'259510","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3472986543,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":262037,"num_read_kb":1015395,"num_write":257133,"num_write_kb":1028214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"124'260040","reported_seq":519652,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:56.579674+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:10:56.579674+0000","last_peered":"2023-10-14T01:10:56.579674+0000","last_clean":"2023-10-14T01:10:56.579674+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:10:56.579674+0000","last_undegraded":"2023-10-14T01:10:56.579674+0000","last_fullsized":"2023-10-14T01:10:56.579674+0000","mapping_epoch":13,"log_start":"120'250012","ondisk_log_start":"120'250012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1018,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 325s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3422970327,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":260569,"num_read_kb":1002407,"num_write":253733,"num_write_kb":1014425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"124'272618","reported_seq":542316,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:55.127022+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:10:55.127022+0000","last_peered":"2023-10-14T01:10:55.127022+0000","last_clean":"2023-10-14T01:10:55.127022+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:10:55.127022+0000","last_undegraded":"2023-10-14T01:10:55.127022+0000","last_fullsized":"2023-10-14T01:10:55.127022+0000","mapping_epoch":13,"log_start":"121'262611","ondisk_log_start":"121'262611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":917,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 323s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3632935332,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":270610,"num_read_kb":1057192,"num_write":267320,"num_write_kb":1069106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"124'269989","reported_seq":535595,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:10:55.094192+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:10:55.094192+0000","last_peered":"2023-10-14T01:10:55.094192+0000","last_clean":"2023-10-14T01:10:55.094192+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:10:55.094192+0000","last_undegraded":"2023-10-14T01:10:55.094192+0000","last_fullsized":"2023-10-14T01:10:55.094192+0000","mapping_epoch":13,"log_start":"121'259914","ondisk_log_start":"121'259914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1013,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 321s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3583750497,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":266560,"num_read_kb":1041670,"num_write":264615,"num_write_kb":1058270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"124'274761","reported_seq":538465,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:54.778396+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:10:54.778396+0000","last_peered":"2023-10-14T01:10:54.778396+0000","last_clean":"2023-10-14T01:10:54.778396+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:10:54.778396+0000","last_undegraded":"2023-10-14T01:10:54.778396+0000","last_fullsized":"2023-10-14T01:10:54.778396+0000","mapping_epoch":13,"log_start":"120'264714","ondisk_log_start":"120'264714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3500876212,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":265030,"num_read_kb":1026719,"num_write":258645,"num_write_kb":1034223,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":328,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-10-14T01:10:03.433520+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:10:03.433520+0000","last_peered":"2023-10-14T01:10:03.433520+0000","last_clean":"2023-10-14T01:10:03.433520+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:10:03.433520+0000","last_undegraded":"2023-10-14T01:10:03.433520+0000","last_fullsized":"2023-10-14T01:10:03.433520+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27397667884,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2059539,"num_read_kb":8011098,"num_write":2025210,"num_write_kb":8098757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16587272192,"data_stored":26463357016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80330,"ondisk_log_size":80330,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739221,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4951212,"kb_used_data":3162212,"kb_used_omap":349,"kb_used_meta":1788642,"kb_avail":89420628,"statfs":{"total":96636764160,"available":91566723072,"internally_reserved":0,"allocated":3238105088,"data_stored":5166478267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1831569745},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:10:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.89300000000000002,"15min":0.78400000000000003},"min":{"1min":0.17799999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.7729999999999997,"5min":6.0880000000000001,"15min":16.474},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.94899999999999995,"15min":0.878},"min":{"1min":0.214,"5min":0.187,"15min":0.187},"max":{"1min":1.891,"5min":11.554,"15min":13.326000000000001},"last":7.8499999999999996}]},{"osd":1,"last update":"Sat Oct 14 01:10:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6040000000000001,"5min":1.448,"15min":1.75},"min":{"1min":0.253,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":5.665,"5min":8.3849999999999998,"15min":14.303000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":1.665,"5min":1.611,"15min":1.8360000000000001},"min":{"1min":0.315,"5min":0.28799999999999998,"15min":0.252},"max":{"1min":5.6920000000000002,"5min":11.593999999999999,"15min":14.35},"last":0.35599999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739214,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10182912,"kb_used_data":8097720,"kb_used_omap":1049,"kb_used_meta":2084134,"kb_avail":84188928,"statfs":{"total":96636764160,"available":86209462272,"internally_reserved":0,"allocated":8292065280,"data_stored":13230009678,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1074571,"internal_metadata":2134153845},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.91300000000000003},"min":{"1min":0.216,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":6.6070000000000002,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.875,"5min":0.69399999999999995,"15min":0.70099999999999996},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":6.6429999999999998,"5min":6.6429999999999998,"15min":9.593},"last":0.30599999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.93400000000000005,"15min":0.76100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":5.569,"5min":10.403,"15min":10.403},"last":0.245},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.998,"15min":0.89000000000000001},"min":{"1min":0.25,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":4.9370000000000003,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.38900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739222,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6433764,"kb_used_data":4939032,"kb_used_omap":594,"kb_used_meta":1494125,"kb_avail":87938076,"statfs":{"total":96636764160,"available":90048589824,"internally_reserved":0,"allocated":5057568768,"data_stored":8066508949,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1529984978},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,7,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.49099999999999999,"15min":0.61099999999999999},"min":{"1min":0.223,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":0.58499999999999996,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.61899999999999999,"15min":0.91400000000000003},"min":{"1min":0.28899999999999998,"5min":0.214,"15min":0.191},"max":{"1min":0.63100000000000001,"5min":9.4849999999999994,"15min":9.4849999999999994},"last":0.32500000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:09:51 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.92700000000000005,"15min":1.0469999999999999},"min":{"1min":0.22700000000000001,"5min":0.218,"15min":0.185},"max":{"1min":5.3979999999999997,"5min":6.5030000000000001,"15min":12.622999999999999},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.69399999999999995,"15min":0.91000000000000003},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.20799999999999999},"max":{"1min":0.65900000000000003,"5min":9.5310000000000006,"15min":9.5310000000000006},"last":0.30299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5057089536,"data_stored":8066127731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8292622336,"data_stored":13230664748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3237560320,"data_stored":5166564537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:10:58.556 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.556 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.557 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.557 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.557 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.558 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.558 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.559 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:10:58.559 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:10:59.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.466+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:10:59.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:10:59.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:10:59.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:10:59.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:10:59.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:10:59.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.526+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:10:59.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.526+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:10:59.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:10:59.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:00.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:00.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:00.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:00.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:00.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:00.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:00.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:00.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:00.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:00.094+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:00.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:00.094+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:00.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:00.122+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:01.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:01.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:01.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:01.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:01.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:01.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.306+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:01.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.310+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:01.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:01.994+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:02.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:02.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:02.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:02.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:02.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:02.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:02.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:02.058+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:02.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:02.058+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:02.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:02.058+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:05.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:05.142 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.138+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:05.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:05.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:05.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:05.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.242+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:05.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.242+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:05.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:05.242+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:06.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:06.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:06.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:06.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:06.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:06.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.666+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:06.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.666+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:06.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:06.670+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:06.737 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:11:06.734+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:11:08.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:08.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:08.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:08.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:08.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:08.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:08.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:08.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:08.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:08.674+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:09.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.810+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:09.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:09.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:09.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:09.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:09.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:09.878+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:10.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.666+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:10.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:10.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:10.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:10.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:10.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.690+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:10.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.690+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:10.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.690+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:11.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:10.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:11.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:11.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:11.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:11.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:11.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:11.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:11.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:11.102+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:11.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:11.102+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:11.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:11.106+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:12.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:12.506+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:12.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:12.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:12.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:12.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:12.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:12.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:12.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:12.506+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:13.099 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:13.097+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:14.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.241+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:14.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:14.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:14.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:14.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:14.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:14.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.281+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:14.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.281+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:14.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:14.281+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:15.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.345+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:11:15.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.345+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:11:15.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.345+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:15.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:15.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:15.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:15.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:15.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:15.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.393+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:15.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.393+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:15.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:15.393+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:16.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:16.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:16.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:16.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:16.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:16.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:16.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:16.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:16.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:16.273+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:16.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:16.273+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:16.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:16.273+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:18.560 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:11:18.612 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:18.605+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:11:18.622 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:18.617+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:18.894 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:18.885+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:11:18.914 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:18.909+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:11:18.916 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:18.913+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:19.428 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:11:19.429 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:11:19.510 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2273,"stamp":"2023-10-14T01:11:18.657990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27430628940,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2067586,"num_read_kb":8042755,"num_write":2033292,"num_write_kb":8132957,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80503,"ondisk_log_size":80503,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":20488044,"kb_used_data":16263188,"kb_used_omap":1994,"kb_used_meta":4222837,"kb_avail":262627476,"statfs":{"total":289910292480,"available":268930535424,"internally_reserved":0,"allocated":16653504512,"data_stored":26528762270,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2042618,"internal_metadata":4324185350},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,7,10,12,14,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16961536,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4043,"num_read_kb":16092,"num_write":4152,"num_write_kb":16608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003447"},"pg_stats":[{"pgid":"2.7","version":"124'238823","reported_seq":474961,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.331069+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:11:15.331069+0000","last_peered":"2023-10-14T01:11:15.331069+0000","last_clean":"2023-10-14T01:11:15.331069+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:11:15.331069+0000","last_undegraded":"2023-10-14T01:11:15.331069+0000","last_fullsized":"2023-10-14T01:11:15.331069+0000","mapping_epoch":13,"log_start":"120'228811","ondisk_log_start":"120'228811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3194282167,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":237000,"num_read_kb":933360,"num_write":237767,"num_write_kb":951053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"124'250489","reported_seq":494942,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.309097+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:11:15.309097+0000","last_peered":"2023-10-14T01:11:15.309097+0000","last_clean":"2023-10-14T01:11:15.309097+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:11:15.309097+0000","last_undegraded":"2023-10-14T01:11:15.309097+0000","last_fullsized":"2023-10-14T01:11:15.309097+0000","mapping_epoch":13,"log_start":"121'240408","ondisk_log_start":"121'240408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3218080011,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":246974,"num_read_kb":950657,"num_write":238024,"num_write_kb":951695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"124'255783","reported_seq":508308,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.335730+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:11:15.335730+0000","last_peered":"2023-10-14T01:11:15.335730+0000","last_clean":"2023-10-14T01:11:15.335730+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:11:15.335730+0000","last_undegraded":"2023-10-14T01:11:15.335730+0000","last_fullsized":"2023-10-14T01:11:15.335730+0000","mapping_epoch":13,"log_start":"121'245712","ondisk_log_start":"121'245712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3383427627,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":253540,"num_read_kb":994782,"num_write":250821,"num_write_kb":1003163,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"124'270630","reported_seq":532357,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.329734+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:11:15.329734+0000","last_peered":"2023-10-14T01:11:15.329734+0000","last_clean":"2023-10-14T01:11:15.329734+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:11:15.329734+0000","last_undegraded":"2023-10-14T01:11:15.329734+0000","last_fullsized":"2023-10-14T01:11:15.329734+0000","mapping_epoch":13,"log_start":"121'260610","ondisk_log_start":"121'260610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3477270959,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":263077,"num_read_kb":1019531,"num_write":258181,"num_write_kb":1032406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"124'261106","reported_seq":521737,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:16.916649+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:11:16.916649+0000","last_peered":"2023-10-14T01:11:16.916649+0000","last_clean":"2023-10-14T01:11:16.916649+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:11:16.916649+0000","last_undegraded":"2023-10-14T01:11:16.916649+0000","last_fullsized":"2023-10-14T01:11:16.916649+0000","mapping_epoch":13,"log_start":"121'251012","ondisk_log_start":"121'251012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1081,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 345s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3427045847,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":261561,"num_read_kb":1006343,"num_write":254731,"num_write_kb":1018417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"124'273691","reported_seq":544481,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:15.288542+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:11:15.288542+0000","last_peered":"2023-10-14T01:11:15.288542+0000","last_clean":"2023-10-14T01:11:15.288542+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:11:15.288542+0000","last_undegraded":"2023-10-14T01:11:15.288542+0000","last_fullsized":"2023-10-14T01:11:15.288542+0000","mapping_epoch":13,"log_start":"121'263611","ondisk_log_start":"121'263611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":969,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 343s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3637174692,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":271678,"num_read_kb":1061448,"num_write":268352,"num_write_kb":1073234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"124'271026","reported_seq":537658,"reported_epoch":124,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:15.256838+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:11:15.256838+0000","last_peered":"2023-10-14T01:11:15.256838+0000","last_clean":"2023-10-14T01:11:15.256838+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:11:15.256838+0000","last_undegraded":"2023-10-14T01:11:15.256838+0000","last_fullsized":"2023-10-14T01:11:15.256838+0000","mapping_epoch":13,"log_start":"121'261014","ondisk_log_start":"121'261014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1069,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 341s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3587809633,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":267569,"num_read_kb":1045698,"num_write":265609,"num_write_kb":1062246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"124'275820","reported_seq":540571,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.308836+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:11:15.308836+0000","last_peered":"2023-10-14T01:11:15.308836+0000","last_clean":"2023-10-14T01:11:15.308836+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:11:15.308836+0000","last_undegraded":"2023-10-14T01:11:15.308836+0000","last_fullsized":"2023-10-14T01:11:15.308836+0000","mapping_epoch":13,"log_start":"121'265814","ondisk_log_start":"121'265814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3504947636,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":266061,"num_read_kb":1030827,"num_write":259638,"num_write_kb":1038195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":329,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:10:18.440087+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:10:18.440087+0000","last_peered":"2023-10-14T01:10:18.440087+0000","last_clean":"2023-10-14T01:10:18.440087+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:10:18.440087+0000","last_undegraded":"2023-10-14T01:10:18.440087+0000","last_fullsized":"2023-10-14T01:10:18.440087+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27430038572,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2067460,"num_read_kb":8042646,"num_write":2033123,"num_write_kb":8130409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16651931648,"data_stored":26528016472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80376,"ondisk_log_size":80376,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739225,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5029836,"kb_used_data":3174212,"kb_used_omap":349,"kb_used_meta":1855266,"kb_avail":89342004,"statfs":{"total":96636764160,"available":91486212096,"internally_reserved":0,"allocated":3250393088,"data_stored":5178766267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1899792721},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.73899999999999999,"15min":0.78500000000000003},"min":{"1min":0.193,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":3.411,"5min":5.7729999999999997,"15min":16.474},"last":0.31},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":0.82499999999999996},"min":{"1min":0.223,"5min":0.187,"15min":0.187},"max":{"1min":7.8499999999999996,"5min":11.554,"15min":11.554},"last":0.23200000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.431,"15min":1.6910000000000001},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":9.0570000000000004,"5min":9.0570000000000004,"15min":14.303000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.5940000000000001,"15min":1.7769999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":9.0990000000000002,"5min":11.593999999999999,"15min":14.35},"last":0.34000000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739218,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8933148,"kb_used_data":8130836,"kb_used_omap":1051,"kb_used_meta":801252,"kb_avail":85438692,"statfs":{"total":96636764160,"available":87489220608,"internally_reserved":0,"allocated":8325976064,"data_stored":13263920462,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":820483043},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,8,7,7,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.91300000000000003},"min":{"1min":0.216,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":6.6070000000000002,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.69399999999999995,"15min":0.70099999999999996},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":6.6429999999999998,"5min":6.6429999999999998,"15min":9.593},"last":0.38800000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.93400000000000005,"15min":0.76100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":5.569,"5min":10.403,"15min":10.403},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.998,"15min":0.89000000000000001},"min":{"1min":0.25,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":4.9370000000000003,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.503}]}]},{"osd":0,"up_from":8,"seq":34359739226,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6525060,"kb_used_data":4958140,"kb_used_omap":594,"kb_used_meta":1566317,"kb_avail":87846780,"statfs":{"total":96636764160,"available":89955102720,"internally_reserved":0,"allocated":5077135360,"data_stored":8086075541,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1603909586},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:10:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.58399999999999996},"min":{"1min":0.19500000000000001,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":2.569,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.46999999999999997,"15min":0.85899999999999999},"min":{"1min":0.28699999999999998,"5min":0.214,"15min":0.191},"max":{"1min":2.5859999999999999,"5min":3.6429999999999998,"15min":9.4849999999999994},"last":0.34599999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:10:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.84999999999999998,"15min":1.032},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":5.5250000000000004,"5min":5.5250000000000004,"15min":12.622999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49299999999999999,"15min":0.871},"min":{"1min":0.254,"5min":0.223,"15min":0.20799999999999999},"max":{"1min":1.4870000000000001,"5min":3.681,"15min":9.5310000000000006},"last":0.38600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5076897792,"data_stored":8085935987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8325165056,"data_stored":13263207468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3249868800,"data_stored":5178873017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:11:19.511 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.512 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.512 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.513 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.513 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.513 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.514 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.514 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:19.514 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:11:19.649 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:19.645+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:11:19.659 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:19.657+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:20.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.237+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:20.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.237+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:20.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:20.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:20.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:20.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:20.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.309+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:20.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.309+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:20.318 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:20.309+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:20.633 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:20.629+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:11:20.636 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:20.633+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:11:20.636 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:20.633+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:11:20.637 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:20.633+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:21.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:21.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:21.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:21.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:21.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:21.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.209+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:21.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.209+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:21.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.217+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:21.222 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:21.217+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:11:21.225 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:21.221+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:21.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:21.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:22.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:22.541+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:22.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:22.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:22.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:22.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:22.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:22.545+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:24.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.341+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:24.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:24.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:24.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:24.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:24.425 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:24.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.413+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:24.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.413+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:24.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:24.461+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:25.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:25.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:25.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:25.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:25.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.441+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:25.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.441+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:25.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.469+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:25.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.569+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:11:25.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.569+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:11:25.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.569+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:25.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:25.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:25.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:25.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:25.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:25.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.593+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:25.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.593+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:25.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:25.593+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:25.762 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:25.757+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:11:25.786 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:25.781+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:27.157 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:27.153+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:11:27.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:27.405+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:27.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:27.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:27.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:27.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:27.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:27.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:27.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:27.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:27.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:27.417+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:27.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:27.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:29.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.081+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:29.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:29.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:29.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.101+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:29.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.101+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:29.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.129+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:29.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:29.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.177+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:29.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:29.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:29.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:29.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.809+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:29.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:29.809+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:30.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:30.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:30.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:30.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:30.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:30.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.581+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:30.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.581+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:30.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:30.581+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:31.172 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:31.169+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:11:34.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:34.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:34.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:34.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:34.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:34.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:34.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:34.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:34.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:34.377+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:35.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:35.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:35.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:35.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:35.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:35.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:35.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:35.101+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:36.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:36.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:36.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:36.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:36.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:36.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.109+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:36.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.109+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:36.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.121+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:36.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:36.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:36.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:36.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.925+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:36.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.925+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:36.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:36.925+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:39.516 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:11:40.337 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:11:40.338 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:11:40.385 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2284,"stamp":"2023-10-14T01:11:38.662023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27464253004,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2075849,"num_read_kb":8075643,"num_write":2041494,"num_write_kb":8165765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80647,"ondisk_log_size":80647,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":20825140,"kb_used_data":16328540,"kb_used_omap":1994,"kb_used_meta":4494581,"kb_avail":262290380,"statfs":{"total":289910292480,"available":268585349120,"internally_reserved":0,"allocated":16720424960,"data_stored":26595670505,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2042618,"internal_metadata":4602451206},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,6,11,17,17,10],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16875520,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4054,"num_read_kb":16120,"num_write":4119,"num_write_kb":16476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.226449"},"pg_stats":[{"pgid":"2.7","version":"125'239810","reported_seq":476918,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:35.748139+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:11:35.748139+0000","last_peered":"2023-10-14T01:11:35.748139+0000","last_clean":"2023-10-14T01:11:35.748139+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:11:35.748139+0000","last_undegraded":"2023-10-14T01:11:35.748139+0000","last_fullsized":"2023-10-14T01:11:35.748139+0000","mapping_epoch":13,"log_start":"121'229711","ondisk_log_start":"121'229711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3198324919,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":237971,"num_read_kb":937220,"num_write":238754,"num_write_kb":955001,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'251481","reported_seq":496956,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:35.719964+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:11:35.719964+0000","last_peered":"2023-10-14T01:11:35.719964+0000","last_clean":"2023-10-14T01:11:35.719964+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:11:35.719964+0000","last_undegraded":"2023-10-14T01:11:35.719964+0000","last_fullsized":"2023-10-14T01:11:35.719964+0000","mapping_epoch":13,"log_start":"121'241408","ondisk_log_start":"121'241408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3221958923,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":247983,"num_read_kb":954673,"num_write":238970,"num_write_kb":955479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"125'256803","reported_seq":510317,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:35.708621+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:11:35.708621+0000","last_peered":"2023-10-14T01:11:35.708621+0000","last_clean":"2023-10-14T01:11:35.708621+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:11:35.708621+0000","last_undegraded":"2023-10-14T01:11:35.708621+0000","last_fullsized":"2023-10-14T01:11:35.708621+0000","mapping_epoch":13,"log_start":"121'246712","ondisk_log_start":"121'246712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3387556395,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":254525,"num_read_kb":998706,"num_write":251826,"num_write_kb":1007183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"125'271710","reported_seq":534523,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:35.710696+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:11:35.710696+0000","last_peered":"2023-10-14T01:11:35.710696+0000","last_clean":"2023-10-14T01:11:35.710696+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:11:35.710696+0000","last_undegraded":"2023-10-14T01:11:35.710696+0000","last_fullsized":"2023-10-14T01:11:35.710696+0000","mapping_epoch":13,"log_start":"121'261610","ondisk_log_start":"121'261610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3481543087,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":264148,"num_read_kb":1023791,"num_write":259213,"num_write_kb":1036534,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"125'262146","reported_seq":523843,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:37.066731+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:11:37.066731+0000","last_peered":"2023-10-14T01:11:37.066731+0000","last_clean":"2023-10-14T01:11:37.066731+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:11:37.066731+0000","last_undegraded":"2023-10-14T01:11:37.066731+0000","last_fullsized":"2023-10-14T01:11:37.066731+0000","mapping_epoch":13,"log_start":"121'252112","ondisk_log_start":"121'252112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1140,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 366s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3431088599,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":262598,"num_read_kb":1010471,"num_write":255720,"num_write_kb":1022373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'274818","reported_seq":546681,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:35.566422+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:11:35.566422+0000","last_peered":"2023-10-14T01:11:35.566422+0000","last_clean":"2023-10-14T01:11:35.566422+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:11:35.566422+0000","last_undegraded":"2023-10-14T01:11:35.566422+0000","last_fullsized":"2023-10-14T01:11:35.566422+0000","mapping_epoch":13,"log_start":"121'264811","ondisk_log_start":"121'264811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1021,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 363s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3641516452,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":272734,"num_read_kb":1065664,"num_write":269419,"num_write_kb":1077502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"125'272159","reported_seq":539881,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:35.582170+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:11:35.582170+0000","last_peered":"2023-10-14T01:11:35.582170+0000","last_clean":"2023-10-14T01:11:35.582170+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:11:35.582170+0000","last_undegraded":"2023-10-14T01:11:35.582170+0000","last_fullsized":"2023-10-14T01:11:35.582170+0000","mapping_epoch":13,"log_start":"121'262114","ondisk_log_start":"121'262114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1124,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 361s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3592192353,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":268647,"num_read_kb":1049978,"num_write":266682,"num_write_kb":1066538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"125'276985","reported_seq":542808,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:35.699431+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:11:35.699431+0000","last_peered":"2023-10-14T01:11:35.699431+0000","last_clean":"2023-10-14T01:11:35.699431+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:11:35.699431+0000","last_undegraded":"2023-10-14T01:11:35.699431+0000","last_fullsized":"2023-10-14T01:11:35.699431+0000","mapping_epoch":13,"log_start":"121'266914","ondisk_log_start":"121'266914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3509481908,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":267117,"num_read_kb":1035031,"num_write":260741,"num_write_kb":1042607,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":330,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.305336+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:11:15.305336+0000","last_peered":"2023-10-14T01:11:15.305336+0000","last_clean":"2023-10-14T01:11:15.305336+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:11:15.305336+0000","last_undegraded":"2023-10-14T01:11:15.305336+0000","last_fullsized":"2023-10-14T01:11:15.305336+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27463662636,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2075723,"num_read_kb":8075534,"num_write":2041325,"num_write_kb":8163217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16718979072,"data_stored":26595063896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80520,"ondisk_log_size":80520,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739229,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5097400,"kb_used_data":3187632,"kb_used_omap":349,"kb_used_meta":1909410,"kb_avail":89274440,"statfs":{"total":96636764160,"available":91417026560,"internally_reserved":0,"allocated":3264135168,"data_stored":5192504276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1955236177},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.73899999999999999,"15min":0.78500000000000003},"min":{"1min":0.193,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":3.411,"5min":5.7729999999999997,"15min":16.474},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":0.82499999999999996},"min":{"1min":0.223,"5min":0.187,"15min":0.187},"max":{"1min":7.8499999999999996,"5min":11.554,"15min":11.554},"last":0.20000000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.431,"15min":1.6910000000000001},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":9.0570000000000004,"5min":9.0570000000000004,"15min":14.303000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.5940000000000001,"15min":1.7769999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":9.0990000000000002,"5min":11.593999999999999,"15min":14.35},"last":0.40400000000000003}]}]},{"osd":1,"up_from":8,"seq":34359739222,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9101552,"kb_used_data":8163176,"kb_used_omap":1051,"kb_used_meta":937316,"kb_avail":85270288,"statfs":{"total":96636764160,"available":87316774912,"internally_reserved":0,"allocated":8359092224,"data_stored":13297032551,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":959812579},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,9,10,10,10],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.84499999999999997,"15min":0.91300000000000003},"min":{"1min":0.216,"5min":0.19700000000000001,"15min":0.17199999999999999},"max":{"1min":6.6070000000000002,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.69399999999999995,"15min":0.70099999999999996},"min":{"1min":0.27800000000000002,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":6.6429999999999998,"5min":6.6429999999999998,"15min":9.593},"last":0.31}]},{"osd":2,"last update":"Sat Oct 14 01:10:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.93400000000000005,"15min":0.76100000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":5.569,"5min":10.403,"15min":10.403},"last":0.34200000000000003},{"interface":"front","average":{"1min":1.0569999999999999,"5min":0.998,"15min":0.89000000000000001},"min":{"1min":0.25,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":4.9370000000000003,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.27200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739230,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6626188,"kb_used_data":4977732,"kb_used_omap":594,"kb_used_meta":1647853,"kb_avail":87745652,"statfs":{"total":96636764160,"available":89851547648,"internally_reserved":0,"allocated":5097197568,"data_stored":8106133678,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1687402450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:10:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.58399999999999996},"min":{"1min":0.19500000000000001,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":2.569,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.46999999999999997,"15min":0.85899999999999999},"min":{"1min":0.28699999999999998,"5min":0.214,"15min":0.191},"max":{"1min":2.5859999999999999,"5min":3.6429999999999998,"15min":9.4849999999999994},"last":0.21199999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:10:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.84999999999999998,"15min":1.032},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":5.5250000000000004,"5min":5.5250000000000004,"15min":12.622999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49299999999999999,"15min":0.871},"min":{"1min":0.254,"5min":0.223,"15min":0.20799999999999999},"max":{"1min":1.4870000000000001,"5min":3.681,"15min":9.5310000000000006},"last":0.25}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5096951808,"data_stored":8105990003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8358920192,"data_stored":13296962604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3263107072,"data_stored":5192111289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:11:40.386 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.387 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.387 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.387 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.388 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.388 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.389 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.389 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:11:40.389 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:11:41.024 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:41.021+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:11:41.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.957+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:41.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:41.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:41.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:42.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:42.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.997+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:42.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.997+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:42.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.997+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:42.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:41.997+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:42.442 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:42.441+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:11:42.496 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:42.493+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:43.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:43.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:43.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:43.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:43.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.181+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:43.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.181+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:43.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.181+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:43.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.197+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:43.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:43.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:43.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:43.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:43.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.253+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:43.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.253+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:43.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.253+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:43.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:43.737+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:48.065 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:48.061+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:50.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:50.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:50.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:50.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:50.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:50.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.561+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:50.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.561+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:50.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.561+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:50.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:50.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:50.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:50.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:51.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:51.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:51.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.001+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:51.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.001+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:51.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.001+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:51.381 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:51.377+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:51.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:51.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:51.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:51.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:51.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:51.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.817+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:51.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.817+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:51.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:51.817+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:54.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:54.556+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:54.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:54.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:54.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:54.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:54.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:54.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:54.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:54.624+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:55.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:55.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:55.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:55.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:55.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:55.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:55.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:55.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:55.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:55.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:55.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:55.380+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:56.475 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:56.472+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:11:56.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.500+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:11:56.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.500+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:11:56.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.532+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:56.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:56.540 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:11:56.536+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:11:56.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:56.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:56.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:56.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:56.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.560+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:56.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.560+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:56.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:56.564+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:57.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.788+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:57.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:57.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:57.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:57.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:57.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.804+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:57.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.804+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:57.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:57.804+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:11:59.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.816+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:11:59.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:11:59.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:11:59.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:11:59.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:11:59.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:11:59.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:11:59.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:11:59.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:11:59.872+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:00.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:00.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:00.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:00.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:00.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:00.364+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:00.391 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:12:00.721 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:00.716+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:00.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:00.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:00.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:00.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:00.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:00.760+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:01.190 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:12:01.190 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:12:01.235 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2295,"stamp":"2023-10-14T01:12:00.669296+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27497111116,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2083909,"num_read_kb":8107671,"num_write":2049501,"num_write_kb":8197793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80451,"ondisk_log_size":80451,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":20923412,"kb_used_data":16393780,"kb_used_omap":1994,"kb_used_meta":4527605,"kb_avail":262192108,"statfs":{"total":289910292480,"available":268484718592,"internally_reserved":0,"allocated":16787230720,"data_stored":26662476265,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2042618,"internal_metadata":4636267782},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,6,9,16,9,11,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16281600,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3969,"num_read_kb":15776,"num_write":3961,"num_write_kb":15844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005538"},"pg_stats":[{"pgid":"2.7","version":"125'240714","reported_seq":478748,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:56.167994+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:11:56.167994+0000","last_peered":"2023-10-14T01:11:56.167994+0000","last_clean":"2023-10-14T01:11:56.167994+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:11:56.167994+0000","last_undegraded":"2023-10-14T01:11:56.167994+0000","last_fullsized":"2023-10-14T01:11:56.167994+0000","mapping_epoch":13,"log_start":"121'230711","ondisk_log_start":"121'230711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3202031799,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":238897,"num_read_kb":940892,"num_write":239658,"num_write_kb":958617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'252464","reported_seq":498911,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:56.215404+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:11:56.215404+0000","last_peered":"2023-10-14T01:11:56.215404+0000","last_clean":"2023-10-14T01:11:56.215404+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:11:56.215404+0000","last_undegraded":"2023-10-14T01:11:56.215404+0000","last_fullsized":"2023-10-14T01:11:56.215404+0000","mapping_epoch":13,"log_start":"121'242408","ondisk_log_start":"121'242408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3225870603,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":248944,"num_read_kb":958501,"num_write":239920,"num_write_kb":959279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"125'257847","reported_seq":512367,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:56.188529+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:11:56.188529+0000","last_peered":"2023-10-14T01:11:56.188529+0000","last_clean":"2023-10-14T01:11:56.188529+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:11:56.188529+0000","last_undegraded":"2023-10-14T01:11:56.188529+0000","last_fullsized":"2023-10-14T01:11:56.188529+0000","mapping_epoch":13,"log_start":"121'247812","ondisk_log_start":"121'247812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3391820331,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":255526,"num_read_kb":1002678,"num_write":252859,"num_write_kb":1011315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"125'272715","reported_seq":536592,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:56.184222+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:11:56.184222+0000","last_peered":"2023-10-14T01:11:56.184222+0000","last_clean":"2023-10-14T01:11:56.184222+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:11:56.184222+0000","last_undegraded":"2023-10-14T01:11:56.184222+0000","last_fullsized":"2023-10-14T01:11:56.184222+0000","mapping_epoch":13,"log_start":"121'262710","ondisk_log_start":"121'262710","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3485524399,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":265198,"num_read_kb":1027967,"num_write":260185,"num_write_kb":1040422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"125'263184","reported_seq":525973,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:57.193180+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:11:57.193180+0000","last_peered":"2023-10-14T01:11:57.193180+0000","last_clean":"2023-10-14T01:11:57.193180+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:11:57.193180+0000","last_undegraded":"2023-10-14T01:11:57.193180+0000","last_fullsized":"2023-10-14T01:11:57.193180+0000","mapping_epoch":13,"log_start":"121'253112","ondisk_log_start":"121'253112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1211,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 386s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3435041239,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":263663,"num_read_kb":1014691,"num_write":256686,"num_write_kb":1026237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'275936","reported_seq":548821,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:55.839710+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:11:55.839710+0000","last_peered":"2023-10-14T01:11:55.839710+0000","last_clean":"2023-10-14T01:11:55.839710+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:11:55.839710+0000","last_undegraded":"2023-10-14T01:11:55.839710+0000","last_fullsized":"2023-10-14T01:11:55.839710+0000","mapping_epoch":13,"log_start":"122'265911","ondisk_log_start":"122'265911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1087,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 383s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3645858212,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":273742,"num_read_kb":1069672,"num_write":270482,"num_write_kb":1081754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"125'273267","reported_seq":542029,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:11:55.792201+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:11:55.792201+0000","last_peered":"2023-10-14T01:11:55.792201+0000","last_clean":"2023-10-14T01:11:55.792201+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:11:55.792201+0000","last_undegraded":"2023-10-14T01:11:55.792201+0000","last_fullsized":"2023-10-14T01:11:55.792201+0000","mapping_epoch":13,"log_start":"122'263214","ondisk_log_start":"122'263214","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1184,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 381s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3596534113,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":269662,"num_read_kb":1054010,"num_write":267741,"num_write_kb":1070774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"125'278089","reported_seq":544965,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:56.190052+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:11:56.190052+0000","last_peered":"2023-10-14T01:11:56.190052+0000","last_clean":"2023-10-14T01:11:56.190052+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:11:56.190052+0000","last_undegraded":"2023-10-14T01:11:56.190052+0000","last_fullsized":"2023-10-14T01:11:56.190052+0000","mapping_epoch":13,"log_start":"121'268014","ondisk_log_start":"121'268014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3513840052,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":268151,"num_read_kb":1039151,"num_write":261801,"num_write_kb":1046847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":330,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.305336+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:11:15.305336+0000","last_peered":"2023-10-14T01:11:15.305336+0000","last_clean":"2023-10-14T01:11:15.305336+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:11:15.305336+0000","last_undegraded":"2023-10-14T01:11:15.305336+0000","last_fullsized":"2023-10-14T01:11:15.305336+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27496520748,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2083783,"num_read_kb":8107562,"num_write":2049332,"num_write_kb":8195245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16784576512,"data_stored":26660661336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80324,"ondisk_log_size":80324,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739233,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5123572,"kb_used_data":3199916,"kb_used_omap":349,"kb_used_meta":1923298,"kb_avail":89248268,"statfs":{"total":96636764160,"available":91390226432,"internally_reserved":0,"allocated":3276713984,"data_stored":5205083092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1969457489},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,4,3,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.73899999999999999,"15min":0.78500000000000003},"min":{"1min":0.193,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":3.411,"5min":5.7729999999999997,"15min":16.474},"last":0.248},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.98299999999999998,"15min":0.82499999999999996},"min":{"1min":0.223,"5min":0.187,"15min":0.187},"max":{"1min":7.8499999999999996,"5min":11.554,"15min":11.554},"last":0.55300000000000005}]},{"osd":1,"last update":"Sat Oct 14 01:11:12 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.431,"15min":1.6910000000000001},"min":{"1min":0.23400000000000001,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":9.0570000000000004,"5min":9.0570000000000004,"15min":14.303000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.5940000000000001,"15min":1.7769999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":9.0990000000000002,"5min":11.593999999999999,"15min":14.35},"last":0.48599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739226,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9114148,"kb_used_data":8196244,"kb_used_omap":1051,"kb_used_meta":916836,"kb_avail":85257692,"statfs":{"total":96636764160,"available":87303876608,"internally_reserved":0,"allocated":8392953856,"data_stored":13330894183,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":938841059},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,12,5,7,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:11:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.75600000000000001,"15min":0.90100000000000002},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.17499999999999999},"max":{"1min":3.7530000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.67100000000000004},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":3.786,"5min":6.6429999999999998,"15min":9.593},"last":0.69799999999999995}]},{"osd":2,"last update":"Sat Oct 14 01:11:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.24399999999999999,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":2.73,"5min":8.4749999999999996,"15min":10.403},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.95099999999999996,"15min":0.89600000000000002},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":1.9950000000000001,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.255}]}]},{"osd":0,"up_from":8,"seq":34359739234,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6685692,"kb_used_data":4997620,"kb_used_omap":594,"kb_used_meta":1687469,"kb_avail":87686148,"statfs":{"total":96636764160,"available":89790615552,"internally_reserved":0,"allocated":5117562880,"data_stored":8126498990,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1727969234},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:10:57 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.49199999999999999,"15min":0.58399999999999996},"min":{"1min":0.19500000000000001,"5min":0.182,"15min":0.16900000000000001},"max":{"1min":2.569,"5min":5.8220000000000001,"15min":6.6829999999999998},"last":0.248},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.46999999999999997,"15min":0.85899999999999999},"min":{"1min":0.28699999999999998,"5min":0.214,"15min":0.191},"max":{"1min":2.5859999999999999,"5min":3.6429999999999998,"15min":9.4849999999999994},"last":0.28799999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:10:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.84999999999999998,"15min":1.032},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":5.5250000000000004,"5min":5.5250000000000004,"15min":12.622999999999999},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.49299999999999999,"15min":0.871},"min":{"1min":0.254,"5min":0.223,"15min":0.20799999999999999},"max":{"1min":1.4870000000000001,"5min":3.681,"15min":9.5310000000000006},"last":0.35899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5116493824,"data_stored":8125532019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8391966720,"data_stored":13330009132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3276115968,"data_stored":5205120185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:12:01.236 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.236 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.237 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.237 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.237 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.238 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.238 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.239 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:01.239 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:12:01.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.448+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:12:01.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.448+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:12:01.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.448+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:01.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:01.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:01.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:01.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:01.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:01.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.484+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:01.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.484+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:01.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:01.484+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:01.509 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:01.500+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:02.317 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:02.312+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:12:02.325 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:02.320+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:02.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:02.832+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:02.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:02.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:02.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:02.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:02.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:02.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:02.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:02.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:02.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:02.876+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:02.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:02.888+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:04.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:04.392+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:04.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:04.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:04.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:04.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:04.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:04.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:04.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:04.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:04.476+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:08.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.052+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:12:08.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.052+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:12:08.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:08.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:08.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:08.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:08.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:08.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.128+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:08.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.128+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:08.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.148+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:08.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:08.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:08.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:08.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:08.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:08.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.608+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:08.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.612+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:08.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:08.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:08.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:08.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:08.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:08.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.948+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:08.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.948+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:08.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:08.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:10.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:10.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:10.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:10.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.536+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:10.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.576+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:10.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:10.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:10.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:10.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:10.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.644+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:10.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.644+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:10.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.648+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:10.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:10.900+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:13.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:13.632+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:13.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:13.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:13.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:13.888+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:14.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:14.740+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:14.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:14.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:14.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:14.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:14.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:14.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:14.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:14.764+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:14.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:14.764+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:14.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:14.772+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:14.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:14.868+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:12:14.904 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:14.900+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:19.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:19.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:19.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:19.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:19.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:19.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.456+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:19.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.456+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:19.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:19.456+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:20.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:20.348+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:20.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:20.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:20.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:20.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:20.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:20.380+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:21.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:12:21.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:21.780+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:21.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:21.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:21.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:21.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:21.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:21.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:21.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:21.824+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:21.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:21.824+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:21.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:21.824+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:22.038 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:12:22.039 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:12:22.076 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2305,"stamp":"2023-10-14T01:12:20.672648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27530513996,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2092229,"num_read_kb":8140803,"num_write":2057663,"num_write_kb":8230441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80584,"ondisk_log_size":80584,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21127760,"kb_used_data":16457848,"kb_used_omap":1994,"kb_used_meta":4667893,"kb_avail":261987760,"statfs":{"total":289910292480,"available":268275466240,"internally_reserved":0,"allocated":16852836352,"data_stored":26728081897,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2042618,"internal_metadata":4779922694},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,5,13,6,12,10,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16371712,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4230,"num_read_kb":16852,"num_write":4006,"num_write_kb":16024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001990"},"pg_stats":[{"pgid":"2.7","version":"125'241707","reported_seq":480688,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:12:16.853065+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:12:16.853065+0000","last_peered":"2023-10-14T01:12:16.853065+0000","last_clean":"2023-10-14T01:12:16.853065+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:12:16.853065+0000","last_undegraded":"2023-10-14T01:12:16.853065+0000","last_fullsized":"2023-10-14T01:12:16.853065+0000","mapping_epoch":13,"log_start":"121'231611","ondisk_log_start":"121'231611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3206103223,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":239841,"num_read_kb":944656,"num_write":240651,"num_write_kb":962589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"125'253471","reported_seq":500907,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:12:16.855724+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:12:16.855724+0000","last_peered":"2023-10-14T01:12:16.855724+0000","last_clean":"2023-10-14T01:12:16.855724+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:12:16.855724+0000","last_undegraded":"2023-10-14T01:12:16.855724+0000","last_fullsized":"2023-10-14T01:12:16.855724+0000","mapping_epoch":13,"log_start":"122'243408","ondisk_log_start":"122'243408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3229786379,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":249919,"num_read_kb":962373,"num_write":240876,"num_write_kb":963103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"125'258878","reported_seq":514469,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:12:16.855155+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:12:16.855155+0000","last_peered":"2023-10-14T01:12:16.855155+0000","last_clean":"2023-10-14T01:12:16.855155+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:12:16.855155+0000","last_undegraded":"2023-10-14T01:12:16.855155+0000","last_fullsized":"2023-10-14T01:12:16.855155+0000","mapping_epoch":13,"log_start":"122'248812","ondisk_log_start":"122'248812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3395977771,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":256595,"num_read_kb":1006934,"num_write":253873,"num_write_kb":1015371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"125'273863","reported_seq":538803,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:12:16.818071+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:12:16.818071+0000","last_peered":"2023-10-14T01:12:16.818071+0000","last_clean":"2023-10-14T01:12:16.818071+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:12:16.818071+0000","last_undegraded":"2023-10-14T01:12:16.818071+0000","last_fullsized":"2023-10-14T01:12:16.818071+0000","mapping_epoch":13,"log_start":"122'263810","ondisk_log_start":"122'263810","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3490001327,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":266251,"num_read_kb":1032159,"num_write":261282,"num_write_kb":1044810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"125'264251","reported_seq":528059,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:12:17.341808+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:12:17.341808+0000","last_peered":"2023-10-14T01:12:17.341808+0000","last_clean":"2023-10-14T01:12:17.341808+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:12:17.341808+0000","last_undegraded":"2023-10-14T01:12:17.341808+0000","last_fullsized":"2023-10-14T01:12:17.341808+0000","mapping_epoch":13,"log_start":"122'254212","ondisk_log_start":"122'254212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1303,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 406s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3439083991,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":264654,"num_read_kb":1018635,"num_write":257676,"num_write_kb":1030197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"125'277061","reported_seq":551080,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:12:16.033131+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:12:16.033131+0000","last_peered":"2023-10-14T01:12:16.033131+0000","last_clean":"2023-10-14T01:12:16.033131+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:12:16.033131+0000","last_undegraded":"2023-10-14T01:12:16.033131+0000","last_fullsized":"2023-10-14T01:12:16.033131+0000","mapping_epoch":13,"log_start":"122'267011","ondisk_log_start":"122'267011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1128,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 403s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3650273700,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":274858,"num_read_kb":1074124,"num_write":271562,"num_write_kb":1086074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"125'274350","reported_seq":544225,"reported_epoch":125,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:12:16.001492+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:12:16.001492+0000","last_peered":"2023-10-14T01:12:16.001492+0000","last_clean":"2023-10-14T01:12:16.001492+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:12:16.001492+0000","last_undegraded":"2023-10-14T01:12:16.001492+0000","last_fullsized":"2023-10-14T01:12:16.001492+0000","mapping_epoch":13,"log_start":"122'264314","ondisk_log_start":"122'264314","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1251,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 401s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3600720225,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":270758,"num_read_kb":1058378,"num_write":268762,"num_write_kb":1074858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"125'279168","reported_seq":547136,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:12:16.820747+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:12:16.820747+0000","last_peered":"2023-10-14T01:12:16.820747+0000","last_clean":"2023-10-14T01:12:16.820747+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:12:16.820747+0000","last_undegraded":"2023-10-14T01:12:16.820747+0000","last_fullsized":"2023-10-14T01:12:16.820747+0000","mapping_epoch":13,"log_start":"122'269114","ondisk_log_start":"122'269114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3517977012,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":269227,"num_read_kb":1043435,"num_write":262812,"num_write_kb":1050891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":330,"reported_epoch":124,"state":"active+clean","last_fresh":"2023-10-14T01:11:15.305336+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:11:15.305336+0000","last_peered":"2023-10-14T01:11:15.305336+0000","last_clean":"2023-10-14T01:11:15.305336+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:11:15.305336+0000","last_undegraded":"2023-10-14T01:11:15.305336+0000","last_fullsized":"2023-10-14T01:11:15.305336+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27529923628,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2092103,"num_read_kb":8140694,"num_write":2057494,"num_write_kb":8227893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16851206144,"data_stored":26727290968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80457,"ondisk_log_size":80457,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739237,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5023400,"kb_used_data":3212320,"kb_used_omap":349,"kb_used_meta":1810722,"kb_avail":89348440,"statfs":{"total":96636764160,"available":91492802560,"internally_reserved":0,"allocated":3289415680,"data_stored":5217784788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1854179665},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,1,8,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74199999999999999,"15min":0.78500000000000003},"min":{"1min":0.22,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.0810000000000004,"5min":5.7729999999999997,"15min":16.474},"last":1.3300000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.024,"15min":0.81599999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.187},"max":{"1min":3.5009999999999999,"5min":11.554,"15min":11.554},"last":0.27100000000000002}]},{"osd":1,"last update":"Sat Oct 14 01:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.4179999999999999,"15min":1.6759999999999999},"min":{"1min":0.26700000000000002,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":5.8170000000000002,"5min":9.0570000000000004,"15min":14.303000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":1.8400000000000001,"5min":1.607,"15min":1.7689999999999999},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":5.8499999999999996,"5min":11.593999999999999,"15min":14.35},"last":0.35999999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739230,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9296068,"kb_used_data":8228220,"kb_used_omap":1051,"kb_used_meta":1066788,"kb_avail":85075772,"statfs":{"total":96636764160,"available":87117590528,"internally_reserved":0,"allocated":8425697280,"data_stored":13363637607,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":1092391907},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,0,8,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:11:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.75600000000000001,"15min":0.90100000000000002},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.17499999999999999},"max":{"1min":3.7530000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.67100000000000004},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":3.786,"5min":6.6429999999999998,"15min":9.593},"last":0.36099999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:11:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.24399999999999999,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":2.73,"5min":8.4749999999999996,"15min":10.403},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.95099999999999996,"15min":0.89600000000000002},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":1.9950000000000001,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.25900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739238,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6808292,"kb_used_data":5017308,"kb_used_omap":594,"kb_used_meta":1790381,"kb_avail":87563548,"statfs":{"total":96636764160,"available":89665073152,"internally_reserved":0,"allocated":5137723392,"data_stored":8146659502,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1833351122},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,1,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.45900000000000002,"15min":0.58199999999999996},"min":{"1min":0.248,"5min":0.19500000000000001,"15min":0.16900000000000001},"max":{"1min":2.4790000000000001,"5min":2.569,"15min":6.6829999999999998},"last":1.6240000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.49399999999999999,"15min":0.83299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":1.7130000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.92000000000000004,"15min":1.016},"min":{"1min":0.23000000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":4.21,"5min":5.5250000000000004,"15min":12.622999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.51400000000000001,"15min":0.85699999999999998},"min":{"1min":0.247,"5min":0.22800000000000001,"15min":0.20799999999999999},"max":{"1min":4.9859999999999998,"5min":4.9859999999999998,"15min":9.5310000000000006},"last":0.316}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5136850944,"data_stored":8145889139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8425570304,"data_stored":13363612716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3288784896,"data_stored":5217789113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:12:22.078 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.078 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.078 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.079 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.079 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.080 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.080 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.080 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:22.081 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:12:22.952 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:12:23.010 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:12:23.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:23.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:23.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:23.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:23.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:23.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:23.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:23.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:23.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:23.792+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:23.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:23.792+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:23.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:23.796+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:23.857 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:12:23.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:12:24.595 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:12:24.635 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:12:25.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.348+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:25.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:25.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:25.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:25.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:25.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:25.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.368+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:25.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.368+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:25.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:25.368+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:25.452 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:12:25.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-14T01:12:26.410 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-14T01:12:26.445 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.1 2023-10-14T01:12:26.829 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:26.828+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:26.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:26.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:26.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:26.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:26.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:26.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:26.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:26.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:26.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:26.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:27.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.004+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:27.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.100+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:12:27.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:27.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:27.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:27.301 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.1 on osd.2 to deep-scrub 2023-10-14T01:12:27.344 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.0 2023-10-14T01:12:27.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:27.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:27.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.940+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:27.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:27.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:27.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:27.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:27.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:28.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:28.028+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:28.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:28.028+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:28.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:28.028+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:28.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:28.032+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:28.182 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.0 on osd.2 to deep-scrub 2023-10-14T01:12:28.285 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:12:29.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.040+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:29.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:29.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:29.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:29.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:29.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:29.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.076+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:29.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:29.092+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:29.234 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:12:29.298 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:12:30.649 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:30.648+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:12:30.650 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:30.652+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:32.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:32.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:32.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:32.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:32.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.220+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:32.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.220+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:32.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.376+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:32.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:32.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:32.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:32.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:32.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.444+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:32.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.444+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:32.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:32.790 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:32.788+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:32.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:32.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:33.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:33.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:33.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:33.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.035+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:33.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.035+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:33.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.043+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:33.524 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:33.523+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:12:33.551 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:33.551+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:33.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:33.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:33.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:33.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:33.715+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:34.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:34.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:34.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:34.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:34.908 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:34.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:34.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:34.903+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:34.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:34.903+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:34.909 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:34.903+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:35.447 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:35.447+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:36.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.627+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:36.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:36.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:36.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:36.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:36.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:36.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.667+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:36.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.667+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:36.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:36.671+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:37.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:37.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:37.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:37.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:37.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:37.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.391+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:37.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.391+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:37.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:37.411+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:40.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:40.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:40.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:40.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:40.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:40.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.131+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:40.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.131+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:40.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:40.187+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:42.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:42.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:42.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:42.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:42.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:42.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:42.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:42.779+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:42.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:42.779+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:42.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:42.791+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:43.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.635+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:43.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:43.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:43.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:43.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:43.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:43.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.695+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:43.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.695+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:43.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:43.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:44.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:44.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:44.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:44.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:44.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:44.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:44.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:44.083+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:44.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:44.083+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:44.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:44.087+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:48.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:48.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:48.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:48.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.463+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:48.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:48.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:48.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:48.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:48.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.535+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:48.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:48.535+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:49.299 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:12:49.918 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:12:49.919 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:12:49.998 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2320,"stamp":"2023-10-14T01:12:48.679442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27580067404,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2104368,"num_read_kb":8189119,"num_write":2069773,"num_write_kb":8278881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80454,"ondisk_log_size":80454,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21604768,"kb_used_data":16554944,"kb_used_omap":1994,"kb_used_meta":5047797,"kb_avail":261510752,"statfs":{"total":289910292480,"available":267787010048,"internally_reserved":0,"allocated":16952262656,"data_stored":26827495859,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2042618,"internal_metadata":5168944390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,9,19,14,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17104896,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4203,"num_read_kb":16728,"num_write":4181,"num_write_kb":16724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001583"},"pg_stats":[{"pgid":"2.7","version":"126'243113","reported_seq":483542,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:12:47.387029+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:12:47.387029+0000","last_peered":"2023-10-14T01:12:47.387029+0000","last_clean":"2023-10-14T01:12:47.387029+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:12:47.387029+0000","last_undegraded":"2023-10-14T01:12:47.387029+0000","last_fullsized":"2023-10-14T01:12:47.387029+0000","mapping_epoch":13,"log_start":"122'233111","ondisk_log_start":"122'233111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3211854007,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":241288,"num_read_kb":950428,"num_write":242057,"num_write_kb":968213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"126'254993","reported_seq":503868,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:12:47.420546+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:12:47.420546+0000","last_peered":"2023-10-14T01:12:47.420546+0000","last_clean":"2023-10-14T01:12:47.420546+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:12:47.420546+0000","last_undegraded":"2023-10-14T01:12:47.420546+0000","last_fullsized":"2023-10-14T01:12:47.420546+0000","mapping_epoch":13,"log_start":"122'244908","ondisk_log_start":"122'244908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3235741963,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":251340,"num_read_kb":968021,"num_write":242328,"num_write_kb":968911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"126'260416","reported_seq":517526,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:12:47.432453+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:12:47.432453+0000","last_peered":"2023-10-14T01:12:47.432453+0000","last_clean":"2023-10-14T01:12:47.432453+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:12:47.432453+0000","last_undegraded":"2023-10-14T01:12:47.432453+0000","last_fullsized":"2023-10-14T01:12:47.432453+0000","mapping_epoch":13,"log_start":"122'250412","ondisk_log_start":"122'250412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3402183211,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":258103,"num_read_kb":1012938,"num_write":255387,"num_write_kb":1021427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"126'275500","reported_seq":542011,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:12:47.423046+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:12:47.423046+0000","last_peered":"2023-10-14T01:12:47.423046+0000","last_clean":"2023-10-14T01:12:47.423046+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:12:47.423046+0000","last_undegraded":"2023-10-14T01:12:47.423046+0000","last_fullsized":"2023-10-14T01:12:47.423046+0000","mapping_epoch":13,"log_start":"122'265410","ondisk_log_start":"122'265410","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3496391087,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":267800,"num_read_kb":1038331,"num_write":262848,"num_write_kb":1051074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"126'265857","reported_seq":531199,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:12:47.752257+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:12:47.752257+0000","last_peered":"2023-10-14T01:12:47.752257+0000","last_clean":"2023-10-14T01:12:47.752257+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:12:47.752257+0000","last_undegraded":"2023-10-14T01:12:47.752257+0000","last_fullsized":"2023-10-14T01:12:47.752257+0000","mapping_epoch":13,"log_start":"122'255812","ondisk_log_start":"122'255812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1371,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 436s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3445350871,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":266148,"num_read_kb":1024567,"num_write":259210,"num_write_kb":1036333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"126'278724","reported_seq":554399,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:12:46.333830+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:12:46.333830+0000","last_peered":"2023-10-14T01:12:46.333830+0000","last_clean":"2023-10-14T01:12:46.333830+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:12:46.333830+0000","last_undegraded":"2023-10-14T01:12:46.333830+0000","last_fullsized":"2023-10-14T01:12:46.333830+0000","mapping_epoch":13,"log_start":"122'268711","ondisk_log_start":"122'268711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1206,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 434s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3656810916,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":276486,"num_read_kb":1080584,"num_write":273159,"num_write_kb":1092462,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"126'275964","reported_seq":547439,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:12:46.357423+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:12:46.357423+0000","last_peered":"2023-10-14T01:12:46.357423+0000","last_clean":"2023-10-14T01:12:46.357423+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:12:46.357423+0000","last_undegraded":"2023-10-14T01:12:46.357423+0000","last_fullsized":"2023-10-14T01:12:46.357423+0000","mapping_epoch":13,"log_start":"122'265914","ondisk_log_start":"122'265914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1327,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 432s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3607056737,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":272336,"num_read_kb":1064678,"num_write":270311,"num_write_kb":1081054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"126'280752","reported_seq":550260,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:12:47.433493+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:12:47.433493+0000","last_peered":"2023-10-14T01:12:47.433493+0000","last_clean":"2023-10-14T01:12:47.433493+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:12:47.433493+0000","last_undegraded":"2023-10-14T01:12:47.433493+0000","last_fullsized":"2023-10-14T01:12:47.433493+0000","mapping_epoch":13,"log_start":"122'270714","ondisk_log_start":"122'270714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3524088244,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":270741,"num_read_kb":1049463,"num_write":264304,"num_write_kb":1056859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":331,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:28.469402+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:11:28.469402+0000","last_peered":"2023-10-14T01:11:28.469402+0000","last_clean":"2023-10-14T01:11:28.469402+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:11:28.469402+0000","last_undegraded":"2023-10-14T01:11:28.469402+0000","last_fullsized":"2023-10-14T01:11:28.469402+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27579477036,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2104242,"num_read_kb":8189010,"num_write":2069604,"num_write_kb":8276333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16950407168,"data_stored":26826491992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80327,"ondisk_log_size":80327,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739243,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5113088,"kb_used_data":3230840,"kb_used_omap":349,"kb_used_meta":1881890,"kb_avail":89258752,"statfs":{"total":96636764160,"available":91400962048,"internally_reserved":0,"allocated":3308380160,"data_stored":5236745154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1927055697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,3,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74199999999999999,"15min":0.78500000000000003},"min":{"1min":0.22,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.0810000000000004,"5min":5.7729999999999997,"15min":16.474},"last":6.1120000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.024,"15min":0.81599999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.187},"max":{"1min":3.5009999999999999,"5min":11.554,"15min":11.554},"last":5.5759999999999996}]},{"osd":1,"last update":"Sat Oct 14 01:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.4179999999999999,"15min":1.6759999999999999},"min":{"1min":0.26700000000000002,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":5.8170000000000002,"5min":9.0570000000000004,"15min":14.303000000000001},"last":3.5779999999999998},{"interface":"front","average":{"1min":1.8400000000000001,"5min":1.607,"15min":1.7689999999999999},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":5.8499999999999996,"5min":11.593999999999999,"15min":14.35},"last":3.6150000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739236,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9541696,"kb_used_data":8277680,"kb_used_omap":1051,"kb_used_meta":1262948,"kb_avail":84830144,"statfs":{"total":96636764160,"available":86866067456,"internally_reserved":0,"allocated":8476344320,"data_stored":13414280533,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":1293259747},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:11:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.75600000000000001,"15min":0.90100000000000002},"min":{"1min":0.20399999999999999,"5min":0.19700000000000001,"15min":0.17499999999999999},"max":{"1min":3.7530000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.67100000000000004},"min":{"1min":0.29399999999999998,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":3.786,"5min":6.6429999999999998,"15min":9.593},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:11:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.24399999999999999,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":2.73,"5min":8.4749999999999996,"15min":10.403},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.436,"5min":0.95099999999999996,"15min":0.89600000000000002},"min":{"1min":0.24199999999999999,"5min":0.221,"15min":0.19500000000000001},"max":{"1min":1.9950000000000001,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.38400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739244,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6949984,"kb_used_data":5046424,"kb_used_omap":594,"kb_used_meta":1902957,"kb_avail":87421856,"statfs":{"total":96636764160,"available":89519980544,"internally_reserved":0,"allocated":5167538176,"data_stored":8176470172,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":1948628946},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.45900000000000002,"15min":0.58199999999999996},"min":{"1min":0.248,"5min":0.19500000000000001,"15min":0.16900000000000001},"max":{"1min":2.4790000000000001,"5min":2.569,"15min":6.6829999999999998},"last":4.6619999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.49399999999999999,"15min":0.83299999999999996},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":5.1559999999999997,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":4.718}]},{"osd":2,"last update":"Sat Oct 14 01:11:58 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.92000000000000004,"15min":1.016},"min":{"1min":0.23000000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":4.21,"5min":5.5250000000000004,"15min":12.622999999999999},"last":0.246},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.51400000000000001,"15min":0.85699999999999998},"min":{"1min":0.247,"5min":0.22800000000000001,"15min":0.20799999999999999},"max":{"1min":4.9859999999999998,"5min":4.9859999999999998,"15min":9.5310000000000006},"last":0.30099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5167276032,"data_stored":8176314227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8475389952,"data_stored":13413432364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3307741184,"data_stored":5236745401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:12:49.999 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.000 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.000 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.001 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.001 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.001 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.002 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.002 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:12:50.003 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:12:50.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.179+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:50.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:50.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:50.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.191+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:50.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.191+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:50.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.243+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:50.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:50.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:50.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:50.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:50.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.255+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:50.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.255+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:50.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:50.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:50.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:50.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:50.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:50.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.535+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:50.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.535+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:50.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:50.591+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:51.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.159+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:51.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:51.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:51.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:51.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:51.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:51.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.195+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:51.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.195+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:51.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:51.195+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:54.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:54.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:54.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:54.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:54.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:54.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.103+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:54.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.103+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:54.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:54.103+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:55.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.675+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:55.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:55.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:12:55.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:55.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:55.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:55.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.723+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:55.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.723+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:55.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.723+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:55.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:55.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:55.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:55.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.855+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:55.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.855+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:55.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.863+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:55.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:55.867+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:56.086 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:56.083+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:12:57.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.015+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:57.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:57.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.039+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:57.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:57.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:57.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.043+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:57.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.043+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:57.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.043+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:57.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:57.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:57.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:57.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:57.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.167+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:57.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.175+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:57.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.771+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:57.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:57.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:57.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:57.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:57.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.783+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:57.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.783+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:57.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:57.783+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:12:58.569 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:58.567+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:12:58.600 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:12:58.599+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:12:59.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.699+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:12:59.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:12:59.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:12:59.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:12:59.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:12:59.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.715+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:12:59.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.715+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:12:59.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:12:59.715+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:00.469 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:00.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:00.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:00.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:00.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:00.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.539+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:00.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.539+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:00.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:00.539+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:01.530 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:01.527+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:02.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:02.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:02.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:02.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:02.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:02.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.603+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:02.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:13:02.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:13:02.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:02.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:02.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:02.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:02.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:02.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:02.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.907+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:02.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:02.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:02.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:03.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:03.123+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:03.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:03.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:03.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:03.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:03.159+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:09.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:09.439+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:09.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:09.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:09.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:09.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:09.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:09.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:09.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:09.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:09.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:09.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:09.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:09.491+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:10.003 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:13:10.668 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:13:10.669 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:13:10.704 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2330,"stamp":"2023-10-14T01:13:08.692602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27613478476,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2112475,"num_read_kb":8221379,"num_write":2077939,"num_write_kb":8311545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80411,"ondisk_log_size":80411,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21867496,"kb_used_data":16619216,"kb_used_omap":1994,"kb_used_meta":5246261,"kb_avail":261248024,"statfs":{"total":289910292480,"available":267517976576,"internally_reserved":0,"allocated":17018077184,"data_stored":26893310387,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2042618,"internal_metadata":5372171526},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,4,16,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22892544,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5591,"num_read_kb":22244,"num_write":5592,"num_write_kb":22368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.011929"},"pg_stats":[{"pgid":"2.7","version":"126'244034","reported_seq":485457,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:07.932174+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:13:07.932174+0000","last_peered":"2023-10-14T01:13:07.932174+0000","last_clean":"2023-10-14T01:13:07.932174+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:13:07.932174+0000","last_undegraded":"2023-10-14T01:13:07.932174+0000","last_fullsized":"2023-10-14T01:13:07.932174+0000","mapping_epoch":13,"log_start":"122'234011","ondisk_log_start":"122'234011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3215626423,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":242281,"num_read_kb":954384,"num_write":242978,"num_write_kb":971897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"126'256012","reported_seq":505852,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:07.909649+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:13:07.909649+0000","last_peered":"2023-10-14T01:13:07.909649+0000","last_clean":"2023-10-14T01:13:07.909649+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:13:07.909649+0000","last_undegraded":"2023-10-14T01:13:07.909649+0000","last_fullsized":"2023-10-14T01:13:07.909649+0000","mapping_epoch":13,"log_start":"122'246008","ondisk_log_start":"122'246008","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3239690507,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":252295,"num_read_kb":971829,"num_write":243294,"num_write_kb":972775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"126'261511","reported_seq":519590,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:07.889820+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:13:07.889820+0000","last_peered":"2023-10-14T01:13:07.889820+0000","last_clean":"2023-10-14T01:13:07.889820+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:13:07.889820+0000","last_undegraded":"2023-10-14T01:13:07.889820+0000","last_fullsized":"2023-10-14T01:13:07.889820+0000","mapping_epoch":13,"log_start":"122'251412","ondisk_log_start":"122'251412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3406570027,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":259068,"num_read_kb":1016786,"num_write":256463,"num_write_kb":1025731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"126'276613","reported_seq":544135,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:07.893774+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:13:07.893774+0000","last_peered":"2023-10-14T01:13:07.893774+0000","last_clean":"2023-10-14T01:13:07.893774+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:13:07.893774+0000","last_undegraded":"2023-10-14T01:13:07.893774+0000","last_fullsized":"2023-10-14T01:13:07.893774+0000","mapping_epoch":13,"log_start":"122'266610","ondisk_log_start":"122'266610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3500728751,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":268798,"num_read_kb":1042299,"num_write":263909,"num_write_kb":1055318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"126'266894","reported_seq":533274,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:07.889290+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:13:07.889290+0000","last_peered":"2023-10-14T01:13:07.889290+0000","last_clean":"2023-10-14T01:13:07.889290+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:13:07.889290+0000","last_undegraded":"2023-10-14T01:13:07.889290+0000","last_fullsized":"2023-10-14T01:13:07.889290+0000","mapping_epoch":13,"log_start":"122'256812","ondisk_log_start":"122'256812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1426,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 456s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3449270743,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":267151,"num_read_kb":1028563,"num_write":260171,"num_write_kb":1040177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"126'279846","reported_seq":556624,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:06.743911+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:13:06.743911+0000","last_peered":"2023-10-14T01:13:06.743911+0000","last_clean":"2023-10-14T01:13:06.743911+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:13:06.743911+0000","last_undegraded":"2023-10-14T01:13:06.743911+0000","last_fullsized":"2023-10-14T01:13:06.743911+0000","mapping_epoch":13,"log_start":"123'269811","ondisk_log_start":"123'269811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1281,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 454s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3661152676,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":277573,"num_read_kb":1084912,"num_write":274220,"num_write_kb":1096706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"126'277142","reported_seq":549675,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:06.670928+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:13:06.670928+0000","last_peered":"2023-10-14T01:13:06.670928+0000","last_clean":"2023-10-14T01:13:06.670928+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:13:06.670928+0000","last_undegraded":"2023-10-14T01:13:06.670928+0000","last_fullsized":"2023-10-14T01:13:06.670928+0000","mapping_epoch":13,"log_start":"122'267114","ondisk_log_start":"122'267114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1389,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 452s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3611664737,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":273375,"num_read_kb":1068798,"num_write":271435,"num_write_kb":1085550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"126'281824","reported_seq":552412,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:07.901650+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:13:07.901650+0000","last_peered":"2023-10-14T01:13:07.901650+0000","last_clean":"2023-10-14T01:13:07.901650+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:13:07.901650+0000","last_undegraded":"2023-10-14T01:13:07.901650+0000","last_fullsized":"2023-10-14T01:13:07.901650+0000","mapping_epoch":13,"log_start":"122'271814","ondisk_log_start":"122'271814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3528184244,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":271808,"num_read_kb":1053699,"num_write":265300,"num_write_kb":1060843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":331,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-10-14T01:11:28.469402+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:11:28.469402+0000","last_peered":"2023-10-14T01:11:28.469402+0000","last_clean":"2023-10-14T01:11:28.469402+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:11:28.469402+0000","last_undegraded":"2023-10-14T01:11:28.469402+0000","last_fullsized":"2023-10-14T01:11:28.469402+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27612888108,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2112349,"num_read_kb":8221270,"num_write":2077770,"num_write_kb":8308997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17017098240,"data_stored":26893183064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80284,"ondisk_log_size":80284,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739247,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5157660,"kb_used_data":3243476,"kb_used_omap":349,"kb_used_meta":1913826,"kb_avail":89214180,"statfs":{"total":96636764160,"available":91355320320,"internally_reserved":0,"allocated":3321319424,"data_stored":5249684418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":1959758161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:12:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74199999999999999,"15min":0.78500000000000003},"min":{"1min":0.22,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":5.0810000000000004,"5min":5.7729999999999997,"15min":16.474},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.86199999999999999,"5min":1.024,"15min":0.81599999999999995},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.187},"max":{"1min":3.5009999999999999,"5min":11.554,"15min":11.554},"last":0.26600000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:12:14 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.4179999999999999,"15min":1.6759999999999999},"min":{"1min":0.26700000000000002,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":5.8170000000000002,"5min":9.0570000000000004,"15min":14.303000000000001},"last":2.5609999999999999},{"interface":"front","average":{"1min":1.8400000000000001,"5min":1.607,"15min":1.7689999999999999},"min":{"1min":0.35999999999999999,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":5.8499999999999996,"5min":11.593999999999999,"15min":14.35},"last":2.5990000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739240,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9686076,"kb_used_data":8309684,"kb_used_omap":1051,"kb_used_meta":1375332,"kb_avail":84685764,"statfs":{"total":96636764160,"available":86718222336,"internally_reserved":0,"allocated":8509116416,"data_stored":13447052629,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":1408340963},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,3,10,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.80500000000000005,"15min":0.90600000000000003},"min":{"1min":0.23799999999999999,"5min":0.19700000000000001,"15min":0.17499999999999999},"max":{"1min":3.7930000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.501},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70699999999999996,"15min":0.65400000000000003},"min":{"1min":0.29299999999999998,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":3.8260000000000001,"5min":6.6429999999999998,"15min":9.593},"last":0.52400000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.83399999999999996,"15min":0.76200000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":7.1280000000000001,"5min":8.4749999999999996,"15min":10.403},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.874,"5min":0.97999999999999998,"15min":0.90900000000000003},"min":{"1min":0.251,"5min":0.221,"15min":0.20000000000000001},"max":{"1min":7.1989999999999998,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.46200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739248,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7023760,"kb_used_data":5066056,"kb_used_omap":594,"kb_used_meta":1957101,"kb_avail":87348080,"statfs":{"total":96636764160,"available":89444433920,"internally_reserved":0,"allocated":5187641344,"data_stored":8196573340,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":2004072402},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,0,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.26700000000000002,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":4.6619999999999999,"5min":4.6619999999999999,"15min":6.6829999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.59699999999999998,"15min":0.81100000000000005},"min":{"1min":0.28199999999999997,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":4.718,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":0.34200000000000003}]},{"osd":2,"last update":"Sat Oct 14 01:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86599999999999999,"15min":1.0149999999999999},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":6.0860000000000003,"5min":6.0860000000000003,"15min":12.622999999999999},"last":0.255},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.56200000000000006,"15min":0.82699999999999996},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.20799999999999999},"max":{"1min":6.8979999999999997,"5min":6.8979999999999997,"15min":9.5310000000000006},"last":0.29499999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5187411968,"data_stored":8196450163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8508997632,"data_stored":13447040044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3320688640,"data_stored":5249692857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:13:10.705 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.706 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.706 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.707 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.707 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.707 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.708 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.708 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:10.708 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:13:10.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:10.866+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:10.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:10.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:10.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:10.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:15.661 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:15.658+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:16.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:16.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:16.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:16.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:16.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.890+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:16.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.890+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:16.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.890+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:16.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:16.910+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:17.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:17.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:17.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:17.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:17.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:17.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:17.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:17.618+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:17.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:17.618+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:17.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:17.618+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:20.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:20.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:20.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:20.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:20.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:20.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:20.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:20.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:20.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:20.518+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:20.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:20.518+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:20.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:20.518+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:22.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:22.566+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:22.585 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:22.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:22.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:22.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:22.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:22.594+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:22.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:22.634+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:24.077 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:24.074+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:13:24.103 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:24.102+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:24.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:24.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:24.379 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:24.378+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:24.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:24.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:24.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:24.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.402+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:24.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.402+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:24.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:24.402+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:25.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:25.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:25.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:25.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.522+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:25.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.522+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:25.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:25.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.786+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:13:25.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:25.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:25.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:25.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:25.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:25.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:25.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:25.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:25.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:25.798+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:26.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:26.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:26.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:26.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:26.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:26.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.746+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:26.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.746+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:26.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:26.746+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:28.239 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:28.234+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:13:28.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:28.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:28.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:28.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:28.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:28.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.862+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:28.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.862+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:28.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:28.862+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:29.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:29.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:29.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:29.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:29.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:29.390+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:29.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:29.518+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:30.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:13:30.880 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:30.878+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:13:30.886 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:30.882+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:31.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.642+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:31.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:31.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:31.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.726+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:31.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:31.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:31.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.790+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:31.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.790+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:31.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:31.806+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:31.856 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:13:31.857 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:13:31.900 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2341,"stamp":"2023-10-14T01:13:30.701016+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27647393356,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2120724,"num_read_kb":8254191,"num_write":2086220,"num_write_kb":8344669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80472,"ondisk_log_size":80472,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22254708,"kb_used_data":16684564,"kb_used_omap":1994,"kb_used_meta":5568117,"kb_avail":260860812,"statfs":{"total":289910292480,"available":267121471488,"internally_reserved":0,"allocated":17084993536,"data_stored":26960226739,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2042618,"internal_metadata":5701752070},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,7,12,21,16,13,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17039360,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4100,"num_read_kb":16320,"num_write":4163,"num_write_kb":16652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003469"},"pg_stats":[{"pgid":"2.7","version":"126'245030","reported_seq":487449,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:28.318790+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:13:28.318790+0000","last_peered":"2023-10-14T01:13:28.318790+0000","last_clean":"2023-10-14T01:13:28.318790+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:13:28.318790+0000","last_undegraded":"2023-10-14T01:13:28.318790+0000","last_fullsized":"2023-10-14T01:13:28.318790+0000","mapping_epoch":13,"log_start":"122'235011","ondisk_log_start":"122'235011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3219714231,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":243278,"num_read_kb":958360,"num_write":243974,"num_write_kb":975881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"126'257033","reported_seq":507921,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:28.347427+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:13:28.347427+0000","last_peered":"2023-10-14T01:13:28.347427+0000","last_clean":"2023-10-14T01:13:28.347427+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:13:28.347427+0000","last_undegraded":"2023-10-14T01:13:28.347427+0000","last_fullsized":"2023-10-14T01:13:28.347427+0000","mapping_epoch":13,"log_start":"122'247008","ondisk_log_start":"122'247008","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3243704587,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":253327,"num_read_kb":975937,"num_write":244277,"num_write_kb":976707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"126'262512","reported_seq":521628,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:28.347458+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:13:28.347458+0000","last_peered":"2023-10-14T01:13:28.347458+0000","last_clean":"2023-10-14T01:13:28.347458+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:13:28.347458+0000","last_undegraded":"2023-10-14T01:13:28.347458+0000","last_fullsized":"2023-10-14T01:13:28.347458+0000","mapping_epoch":13,"log_start":"123'252412","ondisk_log_start":"123'252412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3410657835,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":260102,"num_read_kb":1020906,"num_write":257451,"num_write_kb":1029683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"126'277731","reported_seq":546308,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:28.354242+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:13:28.354242+0000","last_peered":"2023-10-14T01:13:28.354242+0000","last_clean":"2023-10-14T01:13:28.354242+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:13:28.354242+0000","last_undegraded":"2023-10-14T01:13:28.354242+0000","last_fullsized":"2023-10-14T01:13:28.354242+0000","mapping_epoch":13,"log_start":"124'267710","ondisk_log_start":"124'267710","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3505164719,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":269841,"num_read_kb":1046435,"num_write":264991,"num_write_kb":1059646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"126'267924","reported_seq":535306,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:28.019730+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:13:28.019730+0000","last_peered":"2023-10-14T01:13:28.019730+0000","last_clean":"2023-10-14T01:13:28.019730+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:13:28.019730+0000","last_undegraded":"2023-10-14T01:13:28.019730+0000","last_fullsized":"2023-10-14T01:13:28.019730+0000","mapping_epoch":13,"log_start":"123'257912","ondisk_log_start":"123'257912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1493,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 477s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3453309399,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":268121,"num_read_kb":1032419,"num_write":261157,"num_write_kb":1044121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"126'280987","reported_seq":558838,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:26.943428+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:13:26.943428+0000","last_peered":"2023-10-14T01:13:26.943428+0000","last_clean":"2023-10-14T01:13:26.943428+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:13:26.943428+0000","last_undegraded":"2023-10-14T01:13:26.943428+0000","last_fullsized":"2023-10-14T01:13:26.943428+0000","mapping_epoch":13,"log_start":"124'270911","ondisk_log_start":"124'270911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1341,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 474s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3665621412,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":278634,"num_read_kb":1089132,"num_write":275311,"num_write_kb":1101070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"126'278259","reported_seq":551855,"reported_epoch":126,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:26.935317+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:13:26.935317+0000","last_peered":"2023-10-14T01:13:26.935317+0000","last_clean":"2023-10-14T01:13:26.935317+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:13:26.935317+0000","last_undegraded":"2023-10-14T01:13:26.935317+0000","last_fullsized":"2023-10-14T01:13:26.935317+0000","mapping_epoch":13,"log_start":"124'268215","ondisk_log_start":"124'268215","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1452,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 472s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3616002401,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":274420,"num_read_kb":1072958,"num_write":272500,"num_write_kb":1089810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"126'282962","reported_seq":554634,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:28.345764+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:13:28.345764+0000","last_peered":"2023-10-14T01:13:28.345764+0000","last_clean":"2023-10-14T01:13:28.345764+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:13:28.345764+0000","last_undegraded":"2023-10-14T01:13:28.345764+0000","last_fullsized":"2023-10-14T01:13:28.345764+0000","mapping_epoch":13,"log_start":"124'272914","ondisk_log_start":"124'272914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3532628404,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":272875,"num_read_kb":1057935,"num_write":266390,"num_write_kb":1065203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":332,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:12:43.468378+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:12:43.468378+0000","last_peered":"2023-10-14T01:12:43.468378+0000","last_clean":"2023-10-14T01:12:43.468378+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:12:43.468378+0000","last_undegraded":"2023-10-14T01:12:43.468378+0000","last_fullsized":"2023-10-14T01:12:43.468378+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27646802988,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2120598,"num_read_kb":8254082,"num_write":2086051,"num_write_kb":8342121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17084596224,"data_stored":26960681048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80345,"ondisk_log_size":80345,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739251,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5220532,"kb_used_data":3256364,"kb_used_omap":349,"kb_used_meta":1963810,"kb_avail":89151308,"statfs":{"total":96636764160,"available":91290939392,"internally_reserved":0,"allocated":3334516736,"data_stored":5262881730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":358063,"internal_metadata":2010941777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,9,5,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69499999999999995,"15min":0.80200000000000005},"min":{"1min":0.20799999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":6.1120000000000001,"5min":6.1120000000000001,"15min":16.474},"last":0.33300000000000002},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.92800000000000005,"15min":0.81899999999999995},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.5759999999999996,"5min":11.554,"15min":11.554},"last":0.25700000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:13:16 2023","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.5800000000000001,"15min":1.5700000000000001},"min":{"1min":0.24099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":6.0270000000000001,"5min":9.0570000000000004,"15min":13.519},"last":5.6580000000000004},{"interface":"front","average":{"1min":1.9390000000000001,"5min":1.77,"15min":1.6639999999999999},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":6.0679999999999996,"5min":11.593999999999999,"15min":13.565},"last":5.6950000000000003}]}]},{"osd":1,"up_from":8,"seq":34359739244,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9896160,"kb_used_data":8342228,"kb_used_omap":1051,"kb_used_meta":1552868,"kb_avail":84475680,"statfs":{"total":96636764160,"available":86503096320,"internally_reserved":0,"allocated":8542441472,"data_stored":13480377685,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":1590137827},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,1,6,9,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.80500000000000005,"15min":0.90600000000000003},"min":{"1min":0.23799999999999999,"5min":0.19700000000000001,"15min":0.17499999999999999},"max":{"1min":3.7930000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":2.8039999999999998},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70699999999999996,"15min":0.65400000000000003},"min":{"1min":0.29299999999999998,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":3.8260000000000001,"5min":6.6429999999999998,"15min":9.593},"last":2.8340000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.83399999999999996,"15min":0.76200000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":7.1280000000000001,"5min":8.4749999999999996,"15min":10.403},"last":2.7559999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.97999999999999998,"15min":0.90900000000000003},"min":{"1min":0.251,"5min":0.221,"15min":0.20000000000000001},"max":{"1min":7.1989999999999998,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.25600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739252,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7138016,"kb_used_data":5085972,"kb_used_omap":594,"kb_used_meta":2051437,"kb_avail":87233824,"statfs":{"total":96636764160,"available":89327435776,"internally_reserved":0,"allocated":5208035328,"data_stored":8216967324,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":608302,"internal_metadata":2100672466},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.26700000000000002,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":4.6619999999999999,"5min":4.6619999999999999,"15min":6.6829999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.59699999999999998,"15min":0.81100000000000005},"min":{"1min":0.28199999999999997,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":4.718,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":0.502}]},{"osd":2,"last update":"Sat Oct 14 01:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86599999999999999,"15min":1.0149999999999999},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":6.0860000000000003,"5min":6.0860000000000003,"15min":12.622999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.56200000000000006,"15min":0.82699999999999996},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.20799999999999999},"max":{"1min":6.8979999999999997,"5min":6.8979999999999997,"15min":9.5310000000000006},"last":0.52800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5207793664,"data_stored":8216831859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8542875648,"data_stored":13480918060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3333926912,"data_stored":5262931129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:13:31.901 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.902 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.902 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.903 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.903 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.904 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.904 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.904 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:31.905 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:13:36.910 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:36.906+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:42.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:42.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:42.534 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:42.530+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:13:42.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:42.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:42.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.574+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:42.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.574+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:42.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.574+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:42.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:42.610+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:44.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.113+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:44.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:44.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:44.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:44.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:44.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:44.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.177+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:44.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.177+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:44.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.189+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:44.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.241+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:44.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.241+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:44.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:44.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:44.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:44.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:44.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.297+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:44.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.297+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:44.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:44.313+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:44.476 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:44.473+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:13:46.048 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:46.045+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:13:46.058 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:46.053+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:13:46.058 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:46.053+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:47.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.161+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:47.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:47.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:47.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:47.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:47.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.213+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:47.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.213+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:47.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.213+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:47.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.241+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:47.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.465+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:47.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:47.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:47.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:47.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:47.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.513+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:47.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.717+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:47.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:47.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:47.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:47.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:47.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:47.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.749+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:47.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.749+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:47.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:47.749+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:50.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:50.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:50.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:50.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:50.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:50.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:50.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:50.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:50.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:50.273+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:50.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:50.273+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:50.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:50.289+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:51.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:13:52.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.253+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:52.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:52.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:52.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:52.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:52.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:52.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.305+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:52.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.305+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:52.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:52.305+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:52.778 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:13:52.779 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:13:52.841 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2353,"stamp":"2023-10-14T01:13:52.704548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27682737740,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2129528,"num_read_kb":8289227,"num_write":2094840,"num_write_kb":8379149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80527,"ondisk_log_size":80527,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21017900,"kb_used_data":16753864,"kb_used_omap":2046,"kb_used_meta":4261953,"kb_avail":262097620,"statfs":{"total":289910292480,"available":268387962880,"internally_reserved":0,"allocated":17155956736,"data_stored":27031177468,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2095546,"internal_metadata":4364240454},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,19,10,16,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16723968,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4118,"num_read_kb":16396,"num_write":4076,"num_write_kb":16304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001671"},"pg_stats":[{"pgid":"2.7","version":"127'246028","reported_seq":489380,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:13:48.804030+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:13:48.804030+0000","last_peered":"2023-10-14T01:13:48.804030+0000","last_clean":"2023-10-14T01:13:48.804030+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:13:48.804030+0000","last_undegraded":"2023-10-14T01:13:48.804030+0000","last_fullsized":"2023-10-14T01:13:48.804030+0000","mapping_epoch":13,"log_start":"123'236011","ondisk_log_start":"123'236011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3223814327,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":244212,"num_read_kb":962084,"num_write":244972,"num_write_kb":979873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"127'257987","reported_seq":509877,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:13:48.781629+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:13:48.781629+0000","last_peered":"2023-10-14T01:13:48.781629+0000","last_clean":"2023-10-14T01:13:48.781629+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:13:48.781629+0000","last_undegraded":"2023-10-14T01:13:48.781629+0000","last_fullsized":"2023-10-14T01:13:48.781629+0000","mapping_epoch":13,"log_start":"124'247908","ondisk_log_start":"124'247908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3247481099,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":254317,"num_read_kb":979869,"num_write":245198,"num_write_kb":980391,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"127'263531","reported_seq":523663,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:13:48.796786+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:13:48.796786+0000","last_peered":"2023-10-14T01:13:48.796786+0000","last_clean":"2023-10-14T01:13:48.796786+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:13:48.796786+0000","last_undegraded":"2023-10-14T01:13:48.796786+0000","last_fullsized":"2023-10-14T01:13:48.796786+0000","mapping_epoch":13,"log_start":"124'253512","ondisk_log_start":"124'253512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3414807083,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":261114,"num_read_kb":1024942,"num_write":258459,"num_write_kb":1033715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"127'278793","reported_seq":548460,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:13:48.774640+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:13:48.774640+0000","last_peered":"2023-10-14T01:13:48.774640+0000","last_clean":"2023-10-14T01:13:48.774640+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:13:48.774640+0000","last_undegraded":"2023-10-14T01:13:48.774640+0000","last_fullsized":"2023-10-14T01:13:48.774640+0000","mapping_epoch":13,"log_start":"124'268710","ondisk_log_start":"124'268710","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3509367215,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":270919,"num_read_kb":1050719,"num_write":266020,"num_write_kb":1063762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"127'268992","reported_seq":537457,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:48.185652+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:13:48.185652+0000","last_peered":"2023-10-14T01:13:48.185652+0000","last_clean":"2023-10-14T01:13:48.185652+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:13:48.185652+0000","last_undegraded":"2023-10-14T01:13:48.185652+0000","last_fullsized":"2023-10-14T01:13:48.185652+0000","mapping_epoch":13,"log_start":"124'258912","ondisk_log_start":"124'258912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1545,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 497s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3457442263,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":269178,"num_read_kb":1036635,"num_write":262160,"num_write_kb":1048133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"127'282360","reported_seq":561619,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:52.221685+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:13:52.221685+0000","last_peered":"2023-10-14T01:13:52.221685+0000","last_clean":"2023-10-14T01:13:52.221685+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:13:52.221685+0000","last_undegraded":"2023-10-14T01:13:52.221685+0000","last_fullsized":"2023-10-14T01:13:52.221685+0000","mapping_epoch":13,"log_start":"124'272312","ondisk_log_start":"124'272312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1412,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 500s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3670978980,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":280020,"num_read_kb":1094656,"num_write":276620,"num_write_kb":1106306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"127'279642","reported_seq":554559,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:13:52.198761+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:13:52.198761+0000","last_peered":"2023-10-14T01:13:52.198761+0000","last_clean":"2023-10-14T01:13:52.198761+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:13:52.198761+0000","last_undegraded":"2023-10-14T01:13:52.198761+0000","last_fullsized":"2023-10-14T01:13:52.198761+0000","mapping_epoch":13,"log_start":"124'269615","ondisk_log_start":"124'269615","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1552,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 498s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3621302625,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":275720,"num_read_kb":1078118,"num_write":273797,"num_write_kb":1094998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"127'284061","reported_seq":556799,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:13:48.788932+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:13:48.788932+0000","last_peered":"2023-10-14T01:13:48.788932+0000","last_clean":"2023-10-14T01:13:48.788932+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:13:48.788932+0000","last_undegraded":"2023-10-14T01:13:48.788932+0000","last_fullsized":"2023-10-14T01:13:48.788932+0000","mapping_epoch":13,"log_start":"124'274014","ondisk_log_start":"124'274014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3536953780,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":273922,"num_read_kb":1062095,"num_write":267445,"num_write_kb":1069423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":333,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:18.115581+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:13:18.115581+0000","last_peered":"2023-10-14T01:13:18.115581+0000","last_clean":"2023-10-14T01:13:18.115581+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:13:18.115581+0000","last_undegraded":"2023-10-14T01:13:18.115581+0000","last_fullsized":"2023-10-14T01:13:18.115581+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27682147372,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2129402,"num_read_kb":8289118,"num_write":2094671,"num_write_kb":8376601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17154207744,"data_stored":27030292568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80400,"ondisk_log_size":80400,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739256,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4695508,"kb_used_data":3272524,"kb_used_omap":351,"kb_used_meta":1422624,"kb_avail":89676332,"statfs":{"total":96636764160,"available":91828563968,"internally_reserved":0,"allocated":3351064576,"data_stored":5279425413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1456767681},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,7,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69499999999999995,"15min":0.80200000000000005},"min":{"1min":0.20799999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":6.1120000000000001,"5min":6.1120000000000001,"15min":16.474},"last":0.27400000000000002},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.92800000000000005,"15min":0.81899999999999995},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.5759999999999996,"5min":11.554,"15min":11.554},"last":0.221}]},{"osd":1,"last update":"Sat Oct 14 01:13:16 2023","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.5800000000000001,"15min":1.5700000000000001},"min":{"1min":0.24099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":6.0270000000000001,"5min":9.0570000000000004,"15min":13.519},"last":6.077},{"interface":"front","average":{"1min":1.9390000000000001,"5min":1.77,"15min":1.6639999999999999},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":6.0679999999999996,"5min":11.593999999999999,"15min":13.565},"last":6.1189999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739248,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10046100,"kb_used_data":8375624,"kb_used_omap":1051,"kb_used_meta":1669412,"kb_avail":84325740,"statfs":{"total":96636764160,"available":86349557760,"internally_reserved":0,"allocated":8576638976,"data_stored":13514571032,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":1709478883},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,8,4,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.80500000000000005,"15min":0.90600000000000003},"min":{"1min":0.23799999999999999,"5min":0.19700000000000001,"15min":0.17499999999999999},"max":{"1min":3.7930000000000001,"5min":14.957000000000001,"15min":14.957000000000001},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.70699999999999996,"15min":0.65400000000000003},"min":{"1min":0.29299999999999998,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":3.8260000000000001,"5min":6.6429999999999998,"15min":9.593},"last":0.378}]},{"osd":2,"last update":"Sat Oct 14 01:12:56 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.83399999999999996,"15min":0.76200000000000001},"min":{"1min":0.23100000000000001,"5min":0.22900000000000001,"15min":0.20899999999999999},"max":{"1min":7.1280000000000001,"5min":8.4749999999999996,"15min":10.403},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.874,"5min":0.97999999999999998,"15min":0.90900000000000003},"min":{"1min":0.251,"5min":0.221,"15min":0.20000000000000001},"max":{"1min":7.1989999999999998,"5min":7.2240000000000002,"15min":9.2200000000000006},"last":0.35099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359739256,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6276292,"kb_used_data":5105716,"kb_used_omap":644,"kb_used_meta":1169915,"kb_avail":88095548,"statfs":{"total":96636764160,"available":90209841152,"internally_reserved":0,"allocated":5228253184,"data_stored":8237181023,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1197993890},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.55400000000000005,"15min":0.61499999999999999},"min":{"1min":0.26700000000000002,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":4.6619999999999999,"5min":4.6619999999999999,"15min":6.6829999999999998},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.59699999999999998,"15min":0.81100000000000005},"min":{"1min":0.28199999999999997,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":4.718,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":0.29599999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:12:59 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.86599999999999999,"15min":1.0149999999999999},"min":{"1min":0.215,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":6.0860000000000003,"5min":6.0860000000000003,"15min":12.622999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.56200000000000006,"15min":0.82699999999999996},"min":{"1min":0.25,"5min":0.22800000000000001,"15min":0.20799999999999999},"max":{"1min":6.8979999999999997,"5min":6.8979999999999997,"15min":9.5310000000000006},"last":0.25800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5227864064,"data_stored":8236902259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8576294912,"data_stored":13514337324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3350048768,"data_stored":5279052985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:13:52.843 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.843 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.844 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.844 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.844 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.845 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.845 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.846 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:13:52.846 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:13:53.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.633+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:53.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:53.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:53.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:53.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:53.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:53.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.697+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:53.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.697+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:53.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:53.697+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:56.167 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:13:56.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:13:57.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:57.993+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:58.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:13:58.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:58.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.009+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:58.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:58.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:58.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.021+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:58.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.021+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:58.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.045+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:13:58.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.321+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:13:58.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.321+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:13:58.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.321+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:13:58.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:13:58.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.345+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:13:58.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:13:58.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:13:58.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.361+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:13:58.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.361+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:13:58.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:13:58.361+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:01.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.349+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:01.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:01.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:01.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:01.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:01.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:01.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:01.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:01.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.421+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:01.458 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:01.453+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:01.509 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:01.505+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:01.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:01.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:01.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:01.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.977+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:01.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:01.977+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:03.265 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:03.261+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:04.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:04.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:04.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:04.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:04.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:04.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.637+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:04.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.637+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:04.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:04.637+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:05.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.113+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:14:05.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.113+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:14:05.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.125+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:05.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:05.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:05.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:05.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:05.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:05.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.177+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:05.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.177+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:05.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.193+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:05.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.881+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:05.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:05.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:05.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:05.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:05.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:05.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.981+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:05.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.981+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:05.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:05.985+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:06.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:06.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:06.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:06.689+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:06.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:06.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:06.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:06.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:06.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:06.701+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:06.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:06.701+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:06.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:06.701+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:07.172 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:07.165+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:08.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.893+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:08.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:08.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:08.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:08.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:08.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:08.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.949+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:08.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.949+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:08.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:08.949+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:09.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:09.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:09.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:09.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:09.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:09.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.521+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:09.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.533+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:09.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:09.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:10.945 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:10.937+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:10.989 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:10.985+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:11.295 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:11.293+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:11.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:11.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:11.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:11.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:11.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:11.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:11.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:11.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:11.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:11.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:11.688 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:11.685+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:11.688 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:11.685+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:12.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:14:13.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:13.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:13.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:13.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:13.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:13.317+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:13.656 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:14:13.657 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:14:13.716 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2363,"stamp":"2023-10-14T01:14:12.707970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27715755596,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2137754,"num_read_kb":8321999,"num_write":2102898,"num_write_kb":8411381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80543,"ondisk_log_size":80543,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21355780,"kb_used_data":16818600,"kb_used_omap":2046,"kb_used_meta":4535105,"kb_avail":261759740,"statfs":{"total":289910292480,"available":268041973760,"internally_reserved":0,"allocated":17222246400,"data_stored":27097467132,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2095546,"internal_metadata":4643948102},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,2,20,14,18,10,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18788352,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4771,"num_read_kb":19000,"num_write":4591,"num_write_kb":18364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002108"},"pg_stats":[{"pgid":"2.7","version":"127'246924","reported_seq":491238,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:09.346013+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:14:09.346013+0000","last_peered":"2023-10-14T01:14:09.346013+0000","last_clean":"2023-10-14T01:14:09.346013+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:14:09.346013+0000","last_undegraded":"2023-10-14T01:14:09.346013+0000","last_fullsized":"2023-10-14T01:14:09.346013+0000","mapping_epoch":13,"log_start":"124'236911","ondisk_log_start":"124'236911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3227496631,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":245173,"num_read_kb":965920,"num_write":245868,"num_write_kb":983457,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"127'258989","reported_seq":511814,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:09.300274+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:14:09.300274+0000","last_peered":"2023-10-14T01:14:09.300274+0000","last_clean":"2023-10-14T01:14:09.300274+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:14:09.300274+0000","last_undegraded":"2023-10-14T01:14:09.300274+0000","last_fullsized":"2023-10-14T01:14:09.300274+0000","mapping_epoch":13,"log_start":"124'248908","ondisk_log_start":"124'248908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3251400971,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":255239,"num_read_kb":983549,"num_write":246152,"num_write_kb":984207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"127'264575","reported_seq":525733,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:09.332557+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:14:09.332557+0000","last_peered":"2023-10-14T01:14:09.332557+0000","last_clean":"2023-10-14T01:14:09.332557+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:14:09.332557+0000","last_undegraded":"2023-10-14T01:14:09.332557+0000","last_fullsized":"2023-10-14T01:14:09.332557+0000","mapping_epoch":13,"log_start":"124'254512","ondisk_log_start":"124'254512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3419017771,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":262139,"num_read_kb":1029030,"num_write":259487,"num_write_kb":1037827,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"127'279937","reported_seq":550629,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:09.286829+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:14:09.286829+0000","last_peered":"2023-10-14T01:14:09.286829+0000","last_clean":"2023-10-14T01:14:09.286829+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:14:09.286829+0000","last_undegraded":"2023-10-14T01:14:09.286829+0000","last_fullsized":"2023-10-14T01:14:09.286829+0000","mapping_epoch":13,"log_start":"124'269910","ondisk_log_start":"124'269910","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3513860527,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":271934,"num_read_kb":1054767,"num_write":267116,"num_write_kb":1068146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"127'269997","reported_seq":539516,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:08.277955+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:14:08.277955+0000","last_peered":"2023-10-14T01:14:08.277955+0000","last_clean":"2023-10-14T01:14:08.277955+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:14:08.277955+0000","last_undegraded":"2023-10-14T01:14:08.277955+0000","last_fullsized":"2023-10-14T01:14:08.277955+0000","mapping_epoch":13,"log_start":"124'259912","ondisk_log_start":"124'259912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1614,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 517s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3461304791,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":270202,"num_read_kb":1040723,"num_write":263102,"num_write_kb":1051901,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"127'283470","reported_seq":563916,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:12.464556+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:14:12.464556+0000","last_peered":"2023-10-14T01:14:12.464556+0000","last_clean":"2023-10-14T01:14:12.464556+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:14:12.464556+0000","last_undegraded":"2023-10-14T01:14:12.464556+0000","last_fullsized":"2023-10-14T01:14:12.464556+0000","mapping_epoch":13,"log_start":"124'273412","ondisk_log_start":"124'273412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1476,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 520s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3675267492,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":281189,"num_read_kb":1099292,"num_write":277671,"num_write_kb":1110510,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"127'280722","reported_seq":556696,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:12.437208+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:14:12.437208+0000","last_peered":"2023-10-14T01:14:12.437208+0000","last_clean":"2023-10-14T01:14:12.437208+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:14:12.437208+0000","last_undegraded":"2023-10-14T01:14:12.437208+0000","last_fullsized":"2023-10-14T01:14:12.437208+0000","mapping_epoch":13,"log_start":"124'270715","ondisk_log_start":"124'270715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1618,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 518s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3625476449,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":276759,"num_read_kb":1082250,"num_write":274817,"num_write_kb":1099078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"127'285196","reported_seq":559022,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:09.283945+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:14:09.283945+0000","last_peered":"2023-10-14T01:14:09.283945+0000","last_clean":"2023-10-14T01:14:09.283945+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:14:09.283945+0000","last_undegraded":"2023-10-14T01:14:09.283945+0000","last_fullsized":"2023-10-14T01:14:09.283945+0000","mapping_epoch":13,"log_start":"124'275114","ondisk_log_start":"124'275114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3541340596,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":274993,"num_read_kb":1066359,"num_write":268516,"num_write_kb":1073707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":333,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:18.115581+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:13:18.115581+0000","last_peered":"2023-10-14T01:13:18.115581+0000","last_clean":"2023-10-14T01:13:18.115581+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:13:18.115581+0000","last_undegraded":"2023-10-14T01:13:18.115581+0000","last_fullsized":"2023-10-14T01:13:18.115581+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27715165228,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2137628,"num_read_kb":8321890,"num_write":2102729,"num_write_kb":8408833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17219919872,"data_stored":27096004696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80416,"ondisk_log_size":80416,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739260,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4756364,"kb_used_data":3284800,"kb_used_omap":351,"kb_used_meta":1471200,"kb_avail":89615476,"statfs":{"total":96636764160,"available":91766247424,"internally_reserved":0,"allocated":3363635200,"data_stored":5291996037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1506509505},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,4,7,5,7,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:13:16 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.69499999999999995,"15min":0.80200000000000005},"min":{"1min":0.20799999999999999,"5min":0.17699999999999999,"15min":0.17299999999999999},"max":{"1min":6.1120000000000001,"5min":6.1120000000000001,"15min":16.474},"last":0.23000000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.92800000000000005,"15min":0.81899999999999995},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.5759999999999996,"5min":11.554,"15min":11.554},"last":0.441}]},{"osd":1,"last update":"Sat Oct 14 01:13:16 2023","interfaces":[{"interface":"back","average":{"1min":1.8939999999999999,"5min":1.5800000000000001,"15min":1.5700000000000001},"min":{"1min":0.24099999999999999,"5min":0.20699999999999999,"15min":0.20200000000000001},"max":{"1min":6.0270000000000001,"5min":9.0570000000000004,"15min":13.519},"last":0.33900000000000002},{"interface":"front","average":{"1min":1.9390000000000001,"5min":1.77,"15min":1.6639999999999999},"min":{"1min":0.317,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":6.0679999999999996,"5min":11.593999999999999,"15min":13.565},"last":0.38200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739252,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10221232,"kb_used_data":8408680,"kb_used_omap":1051,"kb_used_meta":1811492,"kb_avail":84150608,"statfs":{"total":96636764160,"available":86170222592,"internally_reserved":0,"allocated":8610488320,"data_stored":13548420376,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":1854968803},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,6,13,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.63300000000000001,"15min":0.85999999999999999},"min":{"1min":0.27400000000000002,"5min":0.19700000000000001,"15min":0.184},"max":{"1min":2.8039999999999998,"5min":6.6070000000000002,"15min":14.957000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.68700000000000006,"15min":0.65400000000000003},"min":{"1min":0.316,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":2.8340000000000001,"5min":6.6429999999999998,"15min":9.593},"last":0.60599999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":4.016,"5min":7.1280000000000001,"15min":10.403},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.65800000000000003,"15min":0.90700000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":1.427,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.50700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739260,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6378184,"kb_used_data":5125120,"kb_used_omap":644,"kb_used_meta":1252411,"kb_avail":87993656,"statfs":{"total":96636764160,"available":90105503744,"internally_reserved":0,"allocated":5248122880,"data_stored":8257050719,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1282469794},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,10,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.20200000000000001,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":5.7750000000000004,"5min":5.7750000000000004,"15min":6.6829999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.623,"15min":0.78900000000000003},"min":{"1min":0.217,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":2.492,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":0.28499999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":0.95999999999999996,"15min":1.0600000000000001},"min":{"1min":0.23100000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":7.601,"5min":7.601,"15min":12.622999999999999},"last":3.4340000000000002},{"interface":"front","average":{"1min":0.437,"5min":0.54400000000000004,"15min":0.82399999999999995},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.20799999999999999},"max":{"1min":1.0069999999999999,"5min":6.8979999999999997,"15min":9.5310000000000006},"last":1.8620000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5247631360,"data_stored":8256669555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8609579008,"data_stored":13547621420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3362709504,"data_stored":5291713721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:14:13.717 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.718 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.718 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.719 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.719 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.720 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.720 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.720 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:13.721 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:14:14.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:14.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:14.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:14.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:14.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.077+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:14.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.077+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:14.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.077+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:14.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.901+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:14.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:14.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:14.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:14.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:14.989+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:15.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:15.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.013+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:15.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.013+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:15.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.013+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:15.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:15.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.045+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:15.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:15.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.085+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:15.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:15.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:15.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.193+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:15.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:15.261+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:16.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.093+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:16.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:16.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:16.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.105+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:16.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.105+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:16.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:16.293+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:18.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:18.392+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:18.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:18.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:18.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:18.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:18.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:21.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:21.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:21.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:21.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:21.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:21.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:21.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:21.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:21.344+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:21.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:21.356+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:22.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:22.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:22.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:22.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:22.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:22.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.164+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:22.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.164+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:22.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:22.164+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:24.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:24.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:24.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:24.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:24.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:24.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:24.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:24.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:24.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:24.124+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:24.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:24.124+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:24.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:24.152+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:27.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:27.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:27.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:27.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:27.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:27.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.180+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:27.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.180+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:27.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.180+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:27.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.344+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:27.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:27.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:27.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.380+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:27.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:27.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:27.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.808+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:27.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:27.832+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:28.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:28.008+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:28.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:28.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:28.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:28.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:28.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:28.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:28.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:28.032+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:28.484 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:28.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:29.215 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:29.212+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:14:29.260 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:29.256+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:29.303 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:29.296+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:14:29.320 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:29.312+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:29.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.372+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:29.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:29.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:29.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:29.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.432+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:29.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.432+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:29.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.432+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:29.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.464+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:29.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:29.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:29.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:30.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:30.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:30.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.012+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:30.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.012+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:30.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.012+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:30.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.368+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:14:30.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.376+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:30.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:30.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:30.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:30.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:30.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:30.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.384+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:30.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.384+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:30.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.384+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:30.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:30.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:30.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:30.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.976+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:30.983 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:30.976+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:32.123 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:32.120+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:32.133 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:32.128+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:14:32.143 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:32.140+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:32.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:32.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:32.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:32.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:32.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:32.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.540+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:32.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.540+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:32.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:32.540+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:32.762 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:32.756+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:14:32.762 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:32.756+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:32.767 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:32.764+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:14:32.768 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:32.764+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:33.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:33.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:33.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:33.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:33.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.404+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:33.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.404+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:33.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.404+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:33.722 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:14:33.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:33.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:33.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:33.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.980+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:33.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.980+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:33.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.980+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:33.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:33.980+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:34.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.084+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:14:34.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.096+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:34.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:34.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:34.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:34.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:34.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:34.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.168+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:34.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.184+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:34.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.184+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:34.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.488+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:34.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:34.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:34.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.536+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:34.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:34.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:34.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.548+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:34.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.548+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:34.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:34.560+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:34.671 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:14:34.672 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:14:34.737 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2373,"stamp":"2023-10-14T01:14:32.717818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27749174860,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2146126,"num_read_kb":8355267,"num_write":2111050,"num_write_kb":8443989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80492,"ondisk_log_size":80492,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21682428,"kb_used_data":16882852,"kb_used_omap":2046,"kb_used_meta":4797505,"kb_avail":261433092,"statfs":{"total":289910292480,"available":267707486208,"internally_reserved":0,"allocated":17288040448,"data_stored":27163261180,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2095546,"internal_metadata":4912645702},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,13,15,11,7,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18796544,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4707,"num_read_kb":18700,"num_write":4584,"num_write_kb":18336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005643"},"pg_stats":[{"pgid":"2.7","version":"127'247844","reported_seq":493149,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:29.801885+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:14:29.801885+0000","last_peered":"2023-10-14T01:14:29.801885+0000","last_clean":"2023-10-14T01:14:29.801885+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:14:29.801885+0000","last_undegraded":"2023-10-14T01:14:29.801885+0000","last_fullsized":"2023-10-14T01:14:29.801885+0000","mapping_epoch":13,"log_start":"124'237811","ondisk_log_start":"124'237811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3231256759,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":246165,"num_read_kb":969852,"num_write":246788,"num_write_kb":987137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"127'260058","reported_seq":513893,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:29.786223+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:14:29.786223+0000","last_peered":"2023-10-14T01:14:29.786223+0000","last_clean":"2023-10-14T01:14:29.786223+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:14:29.786223+0000","last_undegraded":"2023-10-14T01:14:29.786223+0000","last_fullsized":"2023-10-14T01:14:29.786223+0000","mapping_epoch":13,"log_start":"124'250008","ondisk_log_start":"124'250008","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3255533835,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":256237,"num_read_kb":987529,"num_write":247161,"num_write_kb":988243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"127'265642","reported_seq":527880,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:29.770868+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:14:29.770868+0000","last_peered":"2023-10-14T01:14:29.770868+0000","last_clean":"2023-10-14T01:14:29.770868+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:14:29.770868+0000","last_undegraded":"2023-10-14T01:14:29.770868+0000","last_fullsized":"2023-10-14T01:14:29.770868+0000","mapping_epoch":13,"log_start":"124'255612","ondisk_log_start":"124'255612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3423330859,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":263209,"num_read_kb":1033282,"num_write":260534,"num_write_kb":1042015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"127'281025","reported_seq":552734,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:29.770172+0000","last_change":"2023-10-14T00:16:06.093814+0000","last_active":"2023-10-14T01:14:29.770172+0000","last_peered":"2023-10-14T01:14:29.770172+0000","last_clean":"2023-10-14T01:14:29.770172+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:14:29.770172+0000","last_undegraded":"2023-10-14T01:14:29.770172+0000","last_fullsized":"2023-10-14T01:14:29.770172+0000","mapping_epoch":13,"log_start":"124'271010","ondisk_log_start":"124'271010","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3518079407,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":272935,"num_read_kb":1058747,"num_write":268144,"num_write_kb":1072258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"127'271086","reported_seq":541720,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:28.424121+0000","last_change":"2023-10-14T01:05:30.869741+0000","last_active":"2023-10-14T01:14:28.424121+0000","last_peered":"2023-10-14T01:14:28.424121+0000","last_clean":"2023-10-14T01:14:28.424121+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:14:28.424121+0000","last_undegraded":"2023-10-14T01:14:28.424121+0000","last_fullsized":"2023-10-14T01:14:28.424121+0000","mapping_epoch":13,"log_start":"124'261012","ondisk_log_start":"124'261012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1681,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 537s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3465462231,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":271291,"num_read_kb":1045047,"num_write":264114,"num_write_kb":1055949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"127'284573","reported_seq":566124,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:32.665358+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:14:32.665358+0000","last_peered":"2023-10-14T01:14:32.665358+0000","last_clean":"2023-10-14T01:14:32.665358+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:14:32.665358+0000","last_undegraded":"2023-10-14T01:14:32.665358+0000","last_fullsized":"2023-10-14T01:14:32.665358+0000","mapping_epoch":13,"log_start":"125'274512","ondisk_log_start":"125'274512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1571,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 540s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3679498660,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":282275,"num_read_kb":1103620,"num_write":278704,"num_write_kb":1114642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"127'281790","reported_seq":558863,"reported_epoch":127,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:32.634829+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:14:32.634829+0000","last_peered":"2023-10-14T01:14:32.634829+0000","last_clean":"2023-10-14T01:14:32.634829+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:14:32.634829+0000","last_undegraded":"2023-10-14T01:14:32.634829+0000","last_fullsized":"2023-10-14T01:14:32.634829+0000","mapping_epoch":13,"log_start":"124'271715","ondisk_log_start":"124'271715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1649,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 538s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3629715809,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":277842,"num_read_kb":1086550,"num_write":275855,"num_write_kb":1103230,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"127'286341","reported_seq":561236,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:14:29.749198+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:14:29.749198+0000","last_peered":"2023-10-14T01:14:29.749198+0000","last_clean":"2023-10-14T01:14:29.749198+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:14:29.749198+0000","last_undegraded":"2023-10-14T01:14:29.749198+0000","last_fullsized":"2023-10-14T01:14:29.749198+0000","mapping_epoch":13,"log_start":"124'276314","ondisk_log_start":"124'276314","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3545706932,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":276046,"num_read_kb":1070531,"num_write":269581,"num_write_kb":1077967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":333,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-10-14T01:13:18.115581+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:13:18.115581+0000","last_peered":"2023-10-14T01:13:18.115581+0000","last_clean":"2023-10-14T01:13:18.115581+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:13:18.115581+0000","last_undegraded":"2023-10-14T01:13:18.115581+0000","last_fullsized":"2023-10-14T01:13:18.115581+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27748584492,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2146000,"num_read_kb":8355158,"num_write":2110881,"num_write_kb":8441441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17286541312,"data_stored":27162626136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80365,"ondisk_log_size":80365,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739264,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4818952,"kb_used_data":3297408,"kb_used_omap":351,"kb_used_meta":1521184,"kb_avail":89552888,"statfs":{"total":96636764160,"available":91702157312,"internally_reserved":0,"allocated":3376545792,"data_stored":5304906629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1557693121},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,6,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.70499999999999996,"15min":0.82299999999999995},"min":{"1min":0.20200000000000001,"5min":0.17799999999999999,"15min":0.17299999999999999},"max":{"1min":2.77,"5min":6.1120000000000001,"15min":16.474},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.80200000000000005,"15min":0.80800000000000005},"min":{"1min":0.20100000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":6.0990000000000002,"5min":7.8499999999999996,"15min":11.554},"last":0.432}]},{"osd":1,"last update":"Sat Oct 14 01:14:18 2023","interfaces":[{"interface":"back","average":{"1min":5.048,"5min":2.2480000000000002,"15min":1.804},"min":{"1min":0.253,"5min":0.23400000000000001,"15min":0.20200000000000001},"max":{"1min":42.481999999999999,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":5.1040000000000001,"5min":2.3279999999999998,"15min":1.8979999999999999},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":42.517000000000003,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.39100000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739256,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10389116,"kb_used_data":8440500,"kb_used_omap":1051,"kb_used_meta":1947556,"kb_avail":83982724,"statfs":{"total":96636764160,"available":85998309376,"internally_reserved":0,"allocated":8643072000,"data_stored":13581004056,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":1994298339},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,8,8,1,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.63300000000000001,"15min":0.85999999999999999},"min":{"1min":0.27400000000000002,"5min":0.19700000000000001,"15min":0.184},"max":{"1min":2.8039999999999998,"5min":6.6070000000000002,"15min":14.957000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.68700000000000006,"15min":0.65400000000000003},"min":{"1min":0.316,"5min":0.23699999999999999,"15min":0.21299999999999999},"max":{"1min":2.8340000000000001,"5min":6.6429999999999998,"15min":9.593},"last":0.435}]},{"osd":2,"last update":"Sat Oct 14 01:13:59 2023","interfaces":[{"interface":"back","average":{"1min":1.0069999999999999,"5min":0.76700000000000002,"15min":0.78000000000000003},"min":{"1min":0.23899999999999999,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":4.016,"5min":7.1280000000000001,"15min":10.403},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.65800000000000003,"15min":0.90700000000000003},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":1.427,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.46400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739264,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6474360,"kb_used_data":5144944,"kb_used_omap":644,"kb_used_meta":1328763,"kb_avail":87897480,"statfs":{"total":96636764160,"available":90007019520,"internally_reserved":0,"allocated":5268422656,"data_stored":8277350495,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1360654242},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.20200000000000001,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":5.7750000000000004,"5min":5.7750000000000004,"15min":6.6829999999999998},"last":2.4830000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.623,"15min":0.78900000000000003},"min":{"1min":0.217,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":2.492,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":2.4670000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":0.95999999999999996,"15min":1.0600000000000001},"min":{"1min":0.23100000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":7.601,"5min":7.601,"15min":12.622999999999999},"last":2.5259999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.54400000000000004,"15min":0.82399999999999995},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.20799999999999999},"max":{"1min":1.0069999999999999,"5min":6.8979999999999997,"15min":9.5310000000000006},"last":2.4159999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5268025344,"data_stored":8277063539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8642949120,"data_stored":13580991532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3375566848,"data_stored":5304571065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:14:34.738 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.739 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.739 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.739 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.740 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.740 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.740 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.741 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:14:34.741 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:14:35.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.092+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:14:35.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.100+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:35.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:35.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:35.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:35.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:35.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.240+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:35.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.284+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:14:35.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.304+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:14:35.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.304+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:35.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:35.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:35.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:35.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:35.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:35.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.328+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:35.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.328+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:35.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:35.328+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:35.533 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:14:35.579 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:14:36.523 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:14:36.571 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:14:37.391 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:14:37.439 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:14:37.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.500+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:37.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:37.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:37.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:37.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:37.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:37.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.580+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:37.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.580+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:37.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.604+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:37.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.616+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:14:37.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.624+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:14:37.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.628+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:37.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:37.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:37.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:37.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:37.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:37.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.660+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:37.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.660+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:37.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:37.660+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:38.338 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:14:38.391 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.2 2023-10-14T01:14:39.312 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.2 on osd.0 to deep-scrub 2023-10-14T01:14:39.365 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.1 2023-10-14T01:14:39.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:39.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:39.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:39.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:39.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:39.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:39.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:39.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:39.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:39.920+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:39.931 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:39.920+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:40.362 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.1 on osd.2 to deep-scrub 2023-10-14T01:14:40.433 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.0 2023-10-14T01:14:41.164 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.0 on osd.2 to deep-scrub 2023-10-14T01:14:41.234 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:14:42.098 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:14:42.149 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:14:42.698 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:42.692+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:42.699 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:42.696+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:14:42.699 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:42.696+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:43.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:43.680+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:43.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:43.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:43.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:43.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:43.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:43.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:44.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:44.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:44.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:44.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:44.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:44.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:44.248 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:44.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:44.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:44.244+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:44.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:44.244+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:44.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:44.264+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:46.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.400+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:46.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.404+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:46.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:46.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:46.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:46.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.464+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:46.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.464+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:46.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.464+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:46.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.464+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:46.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:46.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:46.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:46.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:46.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:50.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.432+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:14:50.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.460+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:50.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:50.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.472+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:50.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:50.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:50.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:50.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:50.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:50.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:50.480+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:51.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:51.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:51.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:51.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.100+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:51.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:51.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:14:51.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:51.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:51.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.692+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:51.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.692+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:51.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:51.744+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:52.346 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:52.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:52.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:52.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:52.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:52.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:52.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:52.436+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:52.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:52.436+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:52.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:52.440+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:54.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.471+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:54.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:54.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:54.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:54.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:54.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.495+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:54.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.495+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:54.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:54.495+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:55.446 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:55.443+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:14:55.448 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:14:55.447+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:14:56.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.347+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:56.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:56.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:56.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:56.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:56.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.383+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:56.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.383+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:56.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.383+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:56.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:56.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:56.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:14:56.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:14:56.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.523+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:14:56.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.523+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:14:56.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:56.535+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:14:59.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:59.391+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:14:59.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:59.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:14:59.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:59.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:14:59.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:14:59.443+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:00.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:00.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:00.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:00.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:00.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:00.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.307+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:00.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.307+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:00.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:00.339+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:00.677 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:00.675+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:15:02.151 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:15:02.856 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:15:02.857 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:15:02.858 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:02.855+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:15:02.909 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2389,"stamp":"2023-10-14T01:15:02.722715+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27795562060,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2158194,"num_read_kb":8403203,"num_write":2122363,"num_write_kb":8489241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80556,"ondisk_log_size":80556,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22141852,"kb_used_data":16974780,"kb_used_omap":2046,"kb_used_meta":5164993,"kb_avail":260973668,"statfs":{"total":289910292480,"available":267237036032,"internally_reserved":0,"allocated":17382174720,"data_stored":27257382852,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2095546,"internal_metadata":5288953414},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,5,5,8,17,21,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16838656,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4408,"num_read_kb":17512,"num_write":4105,"num_write_kb":16420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001541"},"pg_stats":[{"pgid":"2.7","version":"128'249225","reported_seq":496064,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:00.635658+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:15:00.635658+0000","last_peered":"2023-10-14T01:15:00.635658+0000","last_clean":"2023-10-14T01:15:00.635658+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:15:00.635658+0000","last_undegraded":"2023-10-14T01:15:00.635658+0000","last_fullsized":"2023-10-14T01:15:00.635658+0000","mapping_epoch":13,"log_start":"124'239211","ondisk_log_start":"124'239211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3236921527,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":247694,"num_read_kb":975920,"num_write":248169,"num_write_kb":992661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"128'261546","reported_seq":516880,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:00.614429+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:15:00.614429+0000","last_peered":"2023-10-14T01:15:00.614429+0000","last_clean":"2023-10-14T01:15:00.614429+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:15:00.614429+0000","last_undegraded":"2023-10-14T01:15:00.614429+0000","last_fullsized":"2023-10-14T01:15:00.614429+0000","mapping_epoch":13,"log_start":"125'251508","ondisk_log_start":"125'251508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3261284619,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":257715,"num_read_kb":993417,"num_write":248565,"num_write_kb":993859,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"128'267189","reported_seq":531085,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:00.601298+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:15:00.601298+0000","last_peered":"2023-10-14T01:15:00.601298+0000","last_clean":"2023-10-14T01:15:00.601298+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:15:00.601298+0000","last_undegraded":"2023-10-14T01:15:00.601298+0000","last_fullsized":"2023-10-14T01:15:00.601298+0000","mapping_epoch":13,"log_start":"125'257112","ondisk_log_start":"125'257112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3429564971,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":264861,"num_read_kb":1039834,"num_write":262053,"num_write_kb":1048091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"128'282619","reported_seq":556003,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:00.588807+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:15:00.588807+0000","last_peered":"2023-10-14T01:15:00.588807+0000","last_clean":"2023-10-14T01:15:00.588807+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:15:00.588807+0000","last_undegraded":"2023-10-14T01:15:00.588807+0000","last_fullsized":"2023-10-14T01:15:00.588807+0000","mapping_epoch":13,"log_start":"125'272610","ondisk_log_start":"125'272610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":30,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 13s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3524219311,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":274583,"num_read_kb":1065283,"num_write":269634,"num_write_kb":1078218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"128'272627","reported_seq":544875,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:14:58.808172+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:14:58.808172+0000","last_peered":"2023-10-14T01:14:58.808172+0000","last_clean":"2023-10-14T01:14:58.808172+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:14:58.808172+0000","last_undegraded":"2023-10-14T01:14:58.808172+0000","last_fullsized":"2023-10-14T01:14:58.808172+0000","mapping_epoch":13,"log_start":"125'262612","ondisk_log_start":"125'262612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3471393239,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":272859,"num_read_kb":1051279,"num_write":265565,"num_write_kb":1061753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"128'285900","reported_seq":568796,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:58.009748+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:14:58.009748+0000","last_peered":"2023-10-14T01:14:58.009748+0000","last_clean":"2023-10-14T01:14:58.009748+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:14:58.009748+0000","last_undegraded":"2023-10-14T01:14:58.009748+0000","last_fullsized":"2023-10-14T01:14:58.009748+0000","mapping_epoch":13,"log_start":"125'275812","ondisk_log_start":"125'275812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1671,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 565s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3684606372,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":283591,"num_read_kb":1108848,"num_write":279952,"num_write_kb":1119634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"128'283115","reported_seq":561481,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:14:58.002962+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:14:58.002962+0000","last_peered":"2023-10-14T01:14:58.002962+0000","last_clean":"2023-10-14T01:14:58.002962+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:14:58.002962+0000","last_undegraded":"2023-10-14T01:14:58.002962+0000","last_fullsized":"2023-10-14T01:14:58.002962+0000","mapping_epoch":13,"log_start":"125'273015","ondisk_log_start":"125'273015","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1714,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 563s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3634938209,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":279111,"num_read_kb":1091586,"num_write":277123,"num_write_kb":1108302,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"128'288002","reported_seq":564524,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:00.592411+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:15:00.592411+0000","last_peered":"2023-10-14T01:15:00.592411+0000","last_clean":"2023-10-14T01:15:00.592411+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:15:00.592411+0000","last_undegraded":"2023-10-14T01:15:00.592411+0000","last_fullsized":"2023-10-14T01:15:00.592411+0000","mapping_epoch":13,"log_start":"125'277914","ondisk_log_start":"125'277914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3552043444,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":277654,"num_read_kb":1076927,"num_write":271133,"num_write_kb":1084175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":334,"reported_epoch":127,"state":"active+clean","last_fresh":"2023-10-14T01:13:43.539194+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:13:43.539194+0000","last_peered":"2023-10-14T01:13:43.539194+0000","last_clean":"2023-10-14T01:13:43.539194+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:13:43.539194+0000","last_undegraded":"2023-10-14T01:13:43.539194+0000","last_fullsized":"2023-10-14T01:13:43.539194+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27794971692,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2158068,"num_read_kb":8403094,"num_write":2122194,"num_write_kb":8486693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17380405248,"data_stored":27256490072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80429,"ondisk_log_size":80429,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739269,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4930436,"kb_used_data":3312440,"kb_used_omap":351,"kb_used_meta":1617632,"kb_avail":89441404,"statfs":{"total":96636764160,"available":91587997696,"internally_reserved":0,"allocated":3391938560,"data_stored":5320295197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1656455873},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,1,2,5,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.70499999999999996,"15min":0.82299999999999995},"min":{"1min":0.20200000000000001,"5min":0.17799999999999999,"15min":0.17299999999999999},"max":{"1min":2.77,"5min":6.1120000000000001,"15min":16.474},"last":3.5339999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.80200000000000005,"15min":0.80800000000000005},"min":{"1min":0.20100000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":6.0990000000000002,"5min":7.8499999999999996,"15min":11.554},"last":0.23899999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:14:18 2023","interfaces":[{"interface":"back","average":{"1min":5.048,"5min":2.2480000000000002,"15min":1.804},"min":{"1min":0.253,"5min":0.23400000000000001,"15min":0.20200000000000001},"max":{"1min":42.481999999999999,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":5.1040000000000001,"5min":2.3279999999999998,"15min":1.8979999999999999},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":42.517000000000003,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.35899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739262,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10609712,"kb_used_data":8488932,"kb_used_omap":1051,"kb_used_meta":2119716,"kb_avail":83762128,"statfs":{"total":96636764160,"available":85772419072,"internally_reserved":0,"allocated":8692666368,"data_stored":13630594224,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1076253,"internal_metadata":2170590179},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,4,6,9,16],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.59999999999999998,"15min":0.85699999999999998},"min":{"1min":0.30199999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.745,"5min":6.6070000000000002,"15min":14.957000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.65400000000000003,"15min":0.66300000000000003},"min":{"1min":0.33600000000000002,"5min":0.27800000000000002,"15min":0.21299999999999999},"max":{"1min":0.77200000000000002,"5min":6.6429999999999998,"15min":9.593},"last":0.42299999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82599999999999996,"15min":0.80300000000000005},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":5.1280000000000001,"5min":7.1280000000000001,"15min":10.403},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.875},"min":{"1min":0.27300000000000002,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":5.0800000000000001,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.36899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739270,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6601704,"kb_used_data":5173408,"kb_used_omap":644,"kb_used_meta":1427643,"kb_avail":87770136,"statfs":{"total":96636764160,"available":89876619264,"internally_reserved":0,"allocated":5297569792,"data_stored":8306493431,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1461907362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.63600000000000001,"15min":0.63100000000000001},"min":{"1min":0.20200000000000001,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":5.7750000000000004,"5min":5.7750000000000004,"15min":6.6829999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.623,"15min":0.78900000000000003},"min":{"1min":0.217,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":2.492,"5min":5.1559999999999997,"15min":9.4849999999999994},"last":0.69999999999999996}]},{"osd":2,"last update":"Sat Oct 14 01:14:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":0.95999999999999996,"15min":1.0600000000000001},"min":{"1min":0.23100000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":7.601,"5min":7.601,"15min":12.622999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.54400000000000004,"15min":0.82399999999999995},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.20799999999999999},"max":{"1min":1.0069999999999999,"5min":6.8979999999999997,"15min":9.5310000000000006},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5297319936,"data_stored":8306358131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8692002816,"data_stored":13630045228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3391082496,"data_stored":5320086713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:15:02.911 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:02.912 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:02.912 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:02.913 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:02.914 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:02.914 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:02.915 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:02.916 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:15:05.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:05.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:05.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:05.311+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:05.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:05.319+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:05.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:05.379+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:05.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:05.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:05.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:05.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:05.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:05.431+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:06.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.603+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:06.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:06.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:06.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:06.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:06.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:06.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.631+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:06.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.631+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:06.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:06.631+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:09.064 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:09.059+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:15:09.074 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:09.071+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:15:09.074 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:09.071+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:15:12.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:12.355+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:12.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:12.355+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:12.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:12.363+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:12.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:12.379+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:13.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:13.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:13.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:13.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:13.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:13.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:13.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:13.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:13.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:13.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:13.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:13.115+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:15.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.043+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:15.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:15.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:15.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:15.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:15.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:15.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.143+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:15.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.143+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:15.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:15.147+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:16.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.687+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:16.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:16.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:16.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.731+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:16.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:16.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:16.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.739+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:16.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.739+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:16.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:16.763+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:18.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.359+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:18.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.367+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:18.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:18.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:18.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.407+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:18.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.407+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:18.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.407+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:18.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.411+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:18.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.535+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:18.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:18.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:18.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:18.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.607+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:18.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.607+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:18.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.607+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:18.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:18.611+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:19.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:19.099+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:19.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:19.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:19.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:19.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:19.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:19.151+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:20.462 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:20.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:20.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:20.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:20.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:20.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:20.547 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:20.543+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:22.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.683+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:22.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:22.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:22.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:22.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:22.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:22.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.743+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:22.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.743+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:22.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:22.743+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:22.917 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:15:23.630 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:15:23.631 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:15:23.713 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2399,"stamp":"2023-10-14T01:15:22.725902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27828633164,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2166616,"num_read_kb":8436759,"num_write":2130447,"num_write_kb":8521577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80509,"ondisk_log_size":80509,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22167848,"kb_used_data":17038864,"kb_used_omap":2133,"kb_used_meta":5126826,"kb_avail":260947672,"statfs":{"total":289910292480,"available":267210416128,"internally_reserved":0,"allocated":17447796736,"data_stored":27323004868,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2184681,"internal_metadata":5249870359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,6,4,8,10,10,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21581824,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5516,"num_read_kb":21976,"num_write":5283,"num_write_kb":21132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001894"},"pg_stats":[{"pgid":"2.7","version":"128'250172","reported_seq":498002,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:21.082743+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:15:21.082743+0000","last_peered":"2023-10-14T01:15:21.082743+0000","last_clean":"2023-10-14T01:15:21.082743+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:15:21.082743+0000","last_undegraded":"2023-10-14T01:15:21.082743+0000","last_fullsized":"2023-10-14T01:15:21.082743+0000","mapping_epoch":13,"log_start":"125'240111","ondisk_log_start":"125'240111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3240792247,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":248684,"num_read_kb":979856,"num_write":249116,"num_write_kb":996449,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"128'262567","reported_seq":518884,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:21.080886+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:15:21.080886+0000","last_peered":"2023-10-14T01:15:21.080886+0000","last_clean":"2023-10-14T01:15:21.080886+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:15:21.080886+0000","last_undegraded":"2023-10-14T01:15:21.080886+0000","last_fullsized":"2023-10-14T01:15:21.080886+0000","mapping_epoch":13,"log_start":"125'252508","ondisk_log_start":"125'252508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3265331467,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":258686,"num_read_kb":997289,"num_write":249553,"num_write_kb":997811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"128'268176","reported_seq":533107,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:21.082729+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:15:21.082729+0000","last_peered":"2023-10-14T01:15:21.082729+0000","last_clean":"2023-10-14T01:15:21.082729+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:15:21.082729+0000","last_undegraded":"2023-10-14T01:15:21.082729+0000","last_fullsized":"2023-10-14T01:15:21.082729+0000","mapping_epoch":13,"log_start":"125'258112","ondisk_log_start":"125'258112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3433542187,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":265892,"num_read_kb":1043950,"num_write":263029,"num_write_kb":1051995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"128'283678","reported_seq":558117,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:21.034469+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:15:21.034469+0000","last_peered":"2023-10-14T01:15:21.034469+0000","last_clean":"2023-10-14T01:15:21.034469+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:15:21.034469+0000","last_undegraded":"2023-10-14T01:15:21.034469+0000","last_fullsized":"2023-10-14T01:15:21.034469+0000","mapping_epoch":13,"log_start":"125'273610","ondisk_log_start":"125'273610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":91,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 33s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3528233391,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":275620,"num_read_kb":1069423,"num_write":270617,"num_write_kb":1082150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"128'273649","reported_seq":546996,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:18.893497+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:15:18.893497+0000","last_peered":"2023-10-14T01:15:18.893497+0000","last_clean":"2023-10-14T01:15:18.893497+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:15:18.893497+0000","last_undegraded":"2023-10-14T01:15:18.893497+0000","last_fullsized":"2023-10-14T01:15:18.893497+0000","mapping_epoch":13,"log_start":"125'263612","ondisk_log_start":"125'263612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3475530199,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":273936,"num_read_kb":1055555,"num_write":266577,"num_write_kb":1065801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"128'287014","reported_seq":571028,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:18.163789+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:15:18.163789+0000","last_peered":"2023-10-14T01:15:18.163789+0000","last_clean":"2023-10-14T01:15:18.163789+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:15:18.163789+0000","last_undegraded":"2023-10-14T01:15:18.163789+0000","last_fullsized":"2023-10-14T01:15:18.163789+0000","mapping_epoch":13,"log_start":"125'277012","ondisk_log_start":"125'277012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1711,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 586s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3689017764,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":284698,"num_read_kb":1113268,"num_write":281028,"num_write_kb":1123938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"128'284240","reported_seq":563773,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:18.166903+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:15:18.166903+0000","last_peered":"2023-10-14T01:15:18.166903+0000","last_clean":"2023-10-14T01:15:18.166903+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:15:18.166903+0000","last_undegraded":"2023-10-14T01:15:18.166903+0000","last_fullsized":"2023-10-14T01:15:18.166903+0000","mapping_epoch":13,"log_start":"125'274215","ondisk_log_start":"125'274215","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1784,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 584s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3639325025,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":280262,"num_read_kb":1096174,"num_write":278191,"num_write_kb":1112574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"128'289080","reported_seq":566682,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:21.024515+0000","last_change":"2023-10-14T00:16:06.748894+0000","last_active":"2023-10-14T01:15:21.024515+0000","last_peered":"2023-10-14T01:15:21.024515+0000","last_clean":"2023-10-14T01:15:21.024515+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:15:21.024515+0000","last_undegraded":"2023-10-14T01:15:21.024515+0000","last_fullsized":"2023-10-14T01:15:21.024515+0000","mapping_epoch":13,"log_start":"125'279014","ondisk_log_start":"125'279014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3556270516,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":278712,"num_read_kb":1081135,"num_write":272167,"num_write_kb":1088311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":335,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:14:53.556834+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:14:53.556834+0000","last_peered":"2023-10-14T01:14:53.556834+0000","last_clean":"2023-10-14T01:14:53.556834+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:14:53.556834+0000","last_undegraded":"2023-10-14T01:14:53.556834+0000","last_fullsized":"2023-10-14T01:14:53.556834+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27828042796,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2166490,"num_read_kb":8436650,"num_write":2130278,"num_write_kb":8519029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17446244352,"data_stored":27322329176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80382,"ondisk_log_size":80382,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739273,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4974980,"kb_used_data":3325052,"kb_used_omap":351,"kb_used_meta":1649568,"kb_avail":89396860,"statfs":{"total":96636764160,"available":91542384640,"internally_reserved":0,"allocated":3404853248,"data_stored":5333209885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1689158337},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,0,2,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:14:18 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.70499999999999996,"15min":0.82299999999999995},"min":{"1min":0.20200000000000001,"5min":0.17799999999999999,"15min":0.17299999999999999},"max":{"1min":2.77,"5min":6.1120000000000001,"15min":16.474},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.80200000000000005,"15min":0.80800000000000005},"min":{"1min":0.20100000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":6.0990000000000002,"5min":7.8499999999999996,"15min":11.554},"last":0.23899999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:14:18 2023","interfaces":[{"interface":"back","average":{"1min":5.048,"5min":2.2480000000000002,"15min":1.804},"min":{"1min":0.253,"5min":0.23400000000000001,"15min":0.20200000000000001},"max":{"1min":42.481999999999999,"5min":42.481999999999999,"15min":42.481999999999999},"last":4.6539999999999999},{"interface":"front","average":{"1min":5.1040000000000001,"5min":2.3279999999999998,"15min":1.8979999999999999},"min":{"1min":0.38200000000000001,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":42.517000000000003,"5min":42.517000000000003,"15min":42.517000000000003},"last":4.6870000000000003}]}]},{"osd":1,"up_from":8,"seq":34359739266,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10495472,"kb_used_data":8521064,"kb_used_omap":1138,"kb_used_meta":1973261,"kb_avail":83876368,"statfs":{"total":96636764160,"available":85889400832,"internally_reserved":0,"allocated":8725569536,"data_stored":13663497392,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1165388,"internal_metadata":2020620212},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,4,2,4,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.59999999999999998,"15min":0.85699999999999998},"min":{"1min":0.30199999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.745,"5min":6.6070000000000002,"15min":14.957000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.65400000000000003,"15min":0.66300000000000003},"min":{"1min":0.33600000000000002,"5min":0.27800000000000002,"15min":0.21299999999999999},"max":{"1min":0.77200000000000002,"5min":6.6429999999999998,"15min":9.593},"last":1.708}]},{"osd":2,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82599999999999996,"15min":0.80300000000000005},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":5.1280000000000001,"5min":7.1280000000000001,"15min":10.403},"last":1.5980000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.875},"min":{"1min":0.27300000000000002,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":5.0800000000000001,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.28799999999999998}]}]},{"osd":0,"up_from":8,"seq":34359739274,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6697396,"kb_used_data":5192748,"kb_used_omap":644,"kb_used_meta":1503995,"kb_avail":87674444,"statfs":{"total":96636764160,"available":89778630656,"internally_reserved":0,"allocated":5317373952,"data_stored":8326297591,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1540091810},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:15:02 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.75900000000000001,"15min":0.65100000000000002},"min":{"1min":0.23300000000000001,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":6.4459999999999997,"5min":6.4459999999999997,"15min":6.6829999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.71099999999999997,"15min":0.77400000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":6.5,"5min":6.5,"15min":9.4849999999999994},"last":0.35399999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.1319999999999999,"15min":1.0920000000000001},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":7.101,"5min":7.601,"15min":12.622999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":1.262,"5min":0.71599999999999997,"15min":0.81200000000000006},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.20799999999999999},"max":{"1min":7.1440000000000001,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":0.32300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5317091328,"data_stored":8326129523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8725094400,"data_stored":13663136812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3404058624,"data_stored":5333062841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:15:23.715 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:23.715 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:23.716 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:23.716 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:23.717 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:23.717 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:23.718 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:23.718 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:15:24.038 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:24.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:24.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:24.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:24.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:24.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.079+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:24.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.079+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:24.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.087+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:24.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:24.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.475+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:24.476 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:24.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.963+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:24.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:24.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:25.001 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:24.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:25.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:25.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:25.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:25.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:25.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:25.015+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:25.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:25.015+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:25.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:25.027+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:27.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:27.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:27.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:27.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:27.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:27.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:27.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:27.271+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:27.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:27.271+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:27.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:27.275+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:27.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:27.275+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:30.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.003+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:30.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:30.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:30.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:30.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.062+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:30.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.062+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:30.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.062+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:30.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.066+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:30.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.134+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:30.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:30.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.142+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:30.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:30.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.162+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:30.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.162+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:30.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.162+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:30.173 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:30.166+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:31.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.454+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:31.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:31.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.466+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:31.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:31.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:31.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.498+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:31.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.498+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:31.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.498+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:31.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:31.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:31.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:32.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:32.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:32.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:32.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:32.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:32.014+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:32.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:32.014+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:34.754 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:34.750+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:15:36.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:36.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:36.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:36.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:36.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:36.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:36.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:36.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:36.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:36.658+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:36.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:36.658+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:36.676 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:36.674+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:37.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.182+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:37.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:37.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:37.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:37.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:37.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.254+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:37.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.254+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:37.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:37.611 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:15:37.602+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:15:37.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:37.613 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:15:37.610+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:15:37.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:37.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:37.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:37.682+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:38.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:38.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:38.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:38.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:38.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.402+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:38.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.402+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:38.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.402+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:38.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:38.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:38.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:38.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:38.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.758+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:38.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.806+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:38.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:38.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.970+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:38.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:38.994+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:39.871 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:39.866+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:15:40.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.230+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:15:40.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.230+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:40.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:40.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:40.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:40.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:40.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:40.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.318+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:40.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.318+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:40.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.318+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:40.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:40.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:40.890+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:43.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.206+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:43.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:43.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:43.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:43.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.302+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:43.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.322+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:43.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:43.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:43.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.354+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:43.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:43.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:43.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:43.510+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:43.719 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:15:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:15:44.536 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:15:44.578 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2409,"stamp":"2023-10-14T01:15:42.729372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27861306956,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2174784,"num_read_kb":8469311,"num_write":2138401,"num_write_kb":8553393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80568,"ondisk_log_size":80568,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22463012,"kb_used_data":17103048,"kb_used_omap":2133,"kb_used_meta":5357802,"kb_avail":260652508,"statfs":{"total":289910292480,"available":266908168192,"internally_reserved":0,"allocated":17513521152,"data_stored":27388729284,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2184681,"internal_metadata":5486389783},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,0,3,4,6,11,13,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21270528,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5327,"num_read_kb":21232,"num_write":5179,"num_write_kb":20716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001971"},"pg_stats":[{"pgid":"2.7","version":"128'251066","reported_seq":499820,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:41.634779+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:15:41.634779+0000","last_peered":"2023-10-14T01:15:41.634779+0000","last_clean":"2023-10-14T01:15:41.634779+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:15:41.634779+0000","last_undegraded":"2023-10-14T01:15:41.634779+0000","last_fullsized":"2023-10-14T01:15:41.634779+0000","mapping_epoch":13,"log_start":"125'241011","ondisk_log_start":"125'241011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3244490935,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":249608,"num_read_kb":983548,"num_write":250010,"num_write_kb":1000025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"128'263630","reported_seq":520938,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:41.634286+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:15:41.634286+0000","last_peered":"2023-10-14T01:15:41.634286+0000","last_clean":"2023-10-14T01:15:41.634286+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:15:41.634286+0000","last_undegraded":"2023-10-14T01:15:41.634286+0000","last_fullsized":"2023-10-14T01:15:41.634286+0000","mapping_epoch":13,"log_start":"125'253608","ondisk_log_start":"125'253608","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3269447947,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":259665,"num_read_kb":1001193,"num_write":250559,"num_write_kb":1001835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"128'269177","reported_seq":535108,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:41.633822+0000","last_change":"2023-10-14T00:16:06.093800+0000","last_active":"2023-10-14T01:15:41.633822+0000","last_peered":"2023-10-14T01:15:41.633822+0000","last_clean":"2023-10-14T01:15:41.633822+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:15:41.633822+0000","last_undegraded":"2023-10-14T01:15:41.633822+0000","last_fullsized":"2023-10-14T01:15:41.633822+0000","mapping_epoch":13,"log_start":"125'259112","ondisk_log_start":"125'259112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3437539883,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":266888,"num_read_kb":1047926,"num_write":264010,"num_write_kb":1055919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"128'284745","reported_seq":560270,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:41.554709+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:15:41.554709+0000","last_peered":"2023-10-14T01:15:41.554709+0000","last_clean":"2023-10-14T01:15:41.554709+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:15:41.554709+0000","last_undegraded":"2023-10-14T01:15:41.554709+0000","last_fullsized":"2023-10-14T01:15:41.554709+0000","mapping_epoch":13,"log_start":"125'274710","ondisk_log_start":"125'274710","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":170,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 54s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3532173743,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":276681,"num_read_kb":1073635,"num_write":271574,"num_write_kb":1085978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"128'274644","reported_seq":549036,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:39.060505+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:15:39.060505+0000","last_peered":"2023-10-14T01:15:39.060505+0000","last_clean":"2023-10-14T01:15:39.060505+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:15:39.060505+0000","last_undegraded":"2023-10-14T01:15:39.060505+0000","last_fullsized":"2023-10-14T01:15:39.060505+0000","mapping_epoch":13,"log_start":"125'264612","ondisk_log_start":"125'264612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3479593431,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":274966,"num_read_kb":1059655,"num_write":267562,"num_write_kb":1069741,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"128'288100","reported_seq":573162,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:38.432540+0000","last_change":"2023-10-14T01:05:32.000194+0000","last_active":"2023-10-14T01:15:38.432540+0000","last_peered":"2023-10-14T01:15:38.432540+0000","last_clean":"2023-10-14T01:15:38.432540+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:15:38.432540+0000","last_undegraded":"2023-10-14T01:15:38.432540+0000","last_fullsized":"2023-10-14T01:15:38.432540+0000","mapping_epoch":13,"log_start":"125'278012","ondisk_log_start":"125'278012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1808,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 606s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3693167012,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":285726,"num_read_kb":1117360,"num_write":282037,"num_write_kb":1127974,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"128'285408","reported_seq":566044,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:38.430684+0000","last_change":"2023-10-14T01:05:34.016359+0000","last_active":"2023-10-14T01:15:38.430684+0000","last_peered":"2023-10-14T01:15:38.430684+0000","last_clean":"2023-10-14T01:15:38.430684+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:15:38.430684+0000","last_undegraded":"2023-10-14T01:15:38.430684+0000","last_fullsized":"2023-10-14T01:15:38.430684+0000","mapping_epoch":13,"log_start":"125'275315","ondisk_log_start":"125'275315","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1846,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 604s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3643892065,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":281348,"num_read_kb":1100510,"num_write":279304,"num_write_kb":1117026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"128'290165","reported_seq":568834,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:15:41.585576+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:15:41.585576+0000","last_peered":"2023-10-14T01:15:41.585576+0000","last_clean":"2023-10-14T01:15:41.585576+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:15:41.585576+0000","last_undegraded":"2023-10-14T01:15:41.585576+0000","last_fullsized":"2023-10-14T01:15:41.585576+0000","mapping_epoch":13,"log_start":"125'280114","ondisk_log_start":"125'280114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3560411572,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":279776,"num_read_kb":1085375,"num_write":273176,"num_write_kb":1092347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":335,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:14:53.556834+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:14:53.556834+0000","last_peered":"2023-10-14T01:14:53.556834+0000","last_clean":"2023-10-14T01:14:53.556834+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:14:53.556834+0000","last_undegraded":"2023-10-14T01:14:53.556834+0000","last_fullsized":"2023-10-14T01:14:53.556834+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27860716588,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2174658,"num_read_kb":8469202,"num_write":2138232,"num_write_kb":8550845,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17511456768,"data_stored":27387541592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":44970,"internal_metadata":0},"log_size":80441,"ondisk_log_size":80441,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739277,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5010328,"kb_used_data":3337676,"kb_used_omap":351,"kb_used_meta":1672288,"kb_avail":89361512,"statfs":{"total":96636764160,"available":91506188288,"internally_reserved":0,"allocated":3417780224,"data_stored":5346136861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1712423617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,3,5,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.79700000000000004},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.17299999999999999},"max":{"1min":3.5339999999999998,"5min":6.1120000000000001,"15min":16.474},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.84599999999999997,"15min":0.83399999999999996},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.3730000000000002,"5min":7.8499999999999996,"15min":11.554},"last":3.8900000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":2.1829999999999998,"5min":2.3639999999999999,"15min":1.8169999999999999},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.20200000000000001},"max":{"1min":10.388,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":2.2290000000000001,"5min":2.4399999999999999,"15min":1.913},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":10.436999999999999,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.39500000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739270,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10645248,"kb_used_data":8552824,"kb_used_omap":1138,"kb_used_meta":2091277,"kb_avail":83726592,"statfs":{"total":96636764160,"available":85736030208,"internally_reserved":0,"allocated":8758091776,"data_stored":13696019632,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1165388,"internal_metadata":2141468596},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,3,1,3,8,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.59999999999999998,"15min":0.85699999999999998},"min":{"1min":0.30199999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.745,"5min":6.6070000000000002,"15min":14.957000000000001},"last":4.7000000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.65400000000000003,"15min":0.66300000000000003},"min":{"1min":0.33600000000000002,"5min":0.27800000000000002,"15min":0.21299999999999999},"max":{"1min":0.77200000000000002,"5min":6.6429999999999998,"15min":9.593},"last":4.7329999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82599999999999996,"15min":0.80300000000000005},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":5.1280000000000001,"5min":7.1280000000000001,"15min":10.403},"last":3.331},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.875},"min":{"1min":0.27300000000000002,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":5.0800000000000001,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":2.5899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739278,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6807436,"kb_used_data":5212548,"kb_used_omap":644,"kb_used_meta":1594235,"kb_avail":87564404,"statfs":{"total":96636764160,"available":89665949696,"internally_reserved":0,"allocated":5337649152,"data_stored":8346572791,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1632497570},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:15:02 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.75900000000000001,"15min":0.65100000000000002},"min":{"1min":0.23300000000000001,"5min":0.19500000000000001,"15min":0.182},"max":{"1min":6.4459999999999997,"5min":6.4459999999999997,"15min":6.6829999999999998},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.71099999999999997,"15min":0.77400000000000002},"min":{"1min":0.221,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":6.5,"5min":6.5,"15min":9.4849999999999994},"last":4.343}]},{"osd":2,"last update":"Sat Oct 14 01:15:02 2023","interfaces":[{"interface":"back","average":{"1min":1.639,"5min":1.1319999999999999,"15min":1.0920000000000001},"min":{"1min":0.22900000000000001,"5min":0.20200000000000001,"15min":0.185},"max":{"1min":7.101,"5min":7.601,"15min":12.622999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":1.262,"5min":0.71599999999999997,"15min":0.81200000000000006},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.20799999999999999},"max":{"1min":7.1440000000000001,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":4.5359999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5336809472,"data_stored":8345847667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8757731328,"data_stored":13695773740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21241,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3416915968,"data_stored":5345920185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:15:44.579 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:44.580 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:44.580 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:44.580 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:44.581 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:44.581 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:44.582 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:15:44.582 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:15:46.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:46.946+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:46.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:46.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:46.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:46.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:46.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:46.946+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:49.914 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:15:49.910+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:15:50.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:50.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:50.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:50.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:50.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:50.658+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:53.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:53.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:53.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:53.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:53.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.550+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:53.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.550+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:53.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.550+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:53.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:53.982+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:54.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:54.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:54.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:54.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:54.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:54.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.050+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:54.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.066+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:54.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.514+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:54.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:54.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:54.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:54.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:54.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:54.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:54.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:54.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:54.566+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:57.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.078+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:15:57.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.078+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:15:57.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.086+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:57.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:57.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:15:57.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:57.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:57.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:57.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.126+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:57.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.126+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:57.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:57.126+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:59.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.206+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:59.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:59.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:59.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:15:59.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:15:59.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.270+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:15:59.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.270+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:15:59.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.282+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:15:59.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.354+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:15:59.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:15:59.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:15:59.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:15:59.542+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:02.553 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:02.550+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:02.559 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:02.558+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:02.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:02.658+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:02.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:02.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:02.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:02.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:02.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:02.750+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:02.923 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:02.918+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:16:02.932 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:02.930+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:02.933 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:02.930+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:03.228 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:03.226+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:03.229 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:03.226+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:04.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:16:05.441 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:16:05.442 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:16:05.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.470+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:05.510 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2420,"stamp":"2023-10-14T01:16:04.735283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27896233548,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2183532,"num_read_kb":8504203,"num_write":2146907,"num_write_kb":8587417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80516,"ondisk_log_size":80516,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21458068,"kb_used_data":17172216,"kb_used_omap":2122,"kb_used_meta":4283701,"kb_avail":261657452,"statfs":{"total":289910292480,"available":267937230848,"internally_reserved":0,"allocated":17584349184,"data_stored":27459557316,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2173927,"internal_metadata":4386509849},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,2,4,6,9,8,4,3,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18300928,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4640,"num_read_kb":18500,"num_write":4445,"num_write_kb":17780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001986"},"pg_stats":[{"pgid":"2.7","version":"128'251963","reported_seq":501613,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:16:02.552298+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:16:02.552298+0000","last_peered":"2023-10-14T01:16:02.552298+0000","last_clean":"2023-10-14T01:16:02.552298+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:16:02.552298+0000","last_undegraded":"2023-10-14T01:16:02.552298+0000","last_fullsized":"2023-10-14T01:16:02.552298+0000","mapping_epoch":13,"log_start":"125'241911","ondisk_log_start":"125'241911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3248132279,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":250504,"num_read_kb":987120,"num_write":250907,"num_write_kb":1003613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"128'264583","reported_seq":522883,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:16:02.561163+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:16:02.561163+0000","last_peered":"2023-10-14T01:16:02.561163+0000","last_clean":"2023-10-14T01:16:02.561163+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:16:02.561163+0000","last_undegraded":"2023-10-14T01:16:02.561163+0000","last_fullsized":"2023-10-14T01:16:02.561163+0000","mapping_epoch":13,"log_start":"125'254508","ondisk_log_start":"125'254508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3273224459,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":260645,"num_read_kb":1005109,"num_write":251470,"num_write_kb":1005479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"128'270151","reported_seq":537034,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:02.564748+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:16:02.564748+0000","last_peered":"2023-10-14T01:16:02.564748+0000","last_clean":"2023-10-14T01:16:02.564748+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:16:02.564748+0000","last_undegraded":"2023-10-14T01:16:02.564748+0000","last_fullsized":"2023-10-14T01:16:02.564748+0000","mapping_epoch":13,"log_start":"125'260112","ondisk_log_start":"125'260112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3441472043,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":267832,"num_read_kb":1051694,"num_write":264970,"num_write_kb":1059759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"128'285875","reported_seq":562479,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:02.504132+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:16:02.504132+0000","last_peered":"2023-10-14T01:16:02.504132+0000","last_clean":"2023-10-14T01:16:02.504132+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:16:02.504132+0000","last_undegraded":"2023-10-14T01:16:02.504132+0000","last_fullsized":"2023-10-14T01:16:02.504132+0000","mapping_epoch":13,"log_start":"126'275810","ondisk_log_start":"126'275810","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":236,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 75s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3536478639,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":277740,"num_read_kb":1077847,"num_write":272617,"num_write_kb":1090150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"128'275875","reported_seq":551575,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:16:04.293649+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:16:04.293649+0000","last_peered":"2023-10-14T01:16:04.293649+0000","last_clean":"2023-10-14T01:16:04.293649+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:16:04.293649+0000","last_undegraded":"2023-10-14T01:16:04.293649+0000","last_fullsized":"2023-10-14T01:16:04.293649+0000","mapping_epoch":13,"log_start":"126'265812","ondisk_log_start":"126'265812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3484586455,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":276250,"num_read_kb":1064771,"num_write":268783,"num_write_kb":1074625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"128'289418","reported_seq":575809,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:16:03.641848+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:16:03.641848+0000","last_peered":"2023-10-14T01:16:03.641848+0000","last_clean":"2023-10-14T01:16:03.641848+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:16:03.641848+0000","last_undegraded":"2023-10-14T01:16:03.641848+0000","last_fullsized":"2023-10-14T01:16:03.641848+0000","mapping_epoch":13,"log_start":"126'279412","ondisk_log_start":"126'279412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3698299300,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":287029,"num_read_kb":1122556,"num_write":283292,"num_write_kb":1132994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"128'286677","reported_seq":568591,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:16:03.633026+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:16:03.633026+0000","last_peered":"2023-10-14T01:16:03.633026+0000","last_clean":"2023-10-14T01:16:03.633026+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:16:03.633026+0000","last_undegraded":"2023-10-14T01:16:03.633026+0000","last_fullsized":"2023-10-14T01:16:03.633026+0000","mapping_epoch":13,"log_start":"126'276615","ondisk_log_start":"126'276615","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3649007969,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":282613,"num_read_kb":1105566,"num_write":280548,"num_write_kb":1122002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"128'291241","reported_seq":570954,"reported_epoch":128,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:02.482787+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:16:02.482787+0000","last_peered":"2023-10-14T01:16:02.482787+0000","last_clean":"2023-10-14T01:16:02.482787+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:16:02.482787+0000","last_undegraded":"2023-10-14T01:16:02.482787+0000","last_fullsized":"2023-10-14T01:16:02.482787+0000","mapping_epoch":13,"log_start":"126'281214","ondisk_log_start":"126'281214","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":76,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 21s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3564442036,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":280793,"num_read_kb":1089431,"num_write":274151,"num_write_kb":1096247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":335,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:14:53.556834+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:14:53.556834+0000","last_peered":"2023-10-14T01:14:53.556834+0000","last_clean":"2023-10-14T01:14:53.556834+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:14:53.556834+0000","last_undegraded":"2023-10-14T01:14:53.556834+0000","last_fullsized":"2023-10-14T01:14:53.556834+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27895643180,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2183406,"num_read_kb":8504094,"num_write":2146738,"num_write_kb":8584869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17582559232,"data_stored":27458644056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80389,"ondisk_log_size":80389,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739282,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5117560,"kb_used_data":3352432,"kb_used_omap":351,"kb_used_meta":1764768,"kb_avail":89254280,"statfs":{"total":96636764160,"available":91396382720,"internally_reserved":0,"allocated":3432890368,"data_stored":5361247005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1807123137},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,3,6],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.79700000000000004},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.17299999999999999},"max":{"1min":3.5339999999999998,"5min":6.1120000000000001,"15min":16.474},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.84599999999999997,"15min":0.83399999999999996},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.3730000000000002,"5min":7.8499999999999996,"15min":11.554},"last":0.29099999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":2.1829999999999998,"5min":2.3639999999999999,"15min":1.8169999999999999},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.20200000000000001},"max":{"1min":10.388,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.309},{"interface":"front","average":{"1min":2.2290000000000001,"5min":2.4399999999999999,"15min":1.913},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":10.436999999999999,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.35899999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739274,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9487728,"kb_used_data":8584292,"kb_used_omap":1127,"kb_used_meta":902296,"kb_avail":84884112,"statfs":{"total":96636764160,"available":86921330688,"internally_reserved":0,"allocated":8790315008,"data_stored":13728242864,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":923951542},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,1,3,6,7,8,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.59999999999999998,"15min":0.85699999999999998},"min":{"1min":0.30199999999999999,"5min":0.20399999999999999,"15min":0.184},"max":{"1min":0.745,"5min":6.6070000000000002,"15min":14.957000000000001},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.65400000000000003,"15min":0.66300000000000003},"min":{"1min":0.33600000000000002,"5min":0.27800000000000002,"15min":0.21299999999999999},"max":{"1min":0.77200000000000002,"5min":6.6429999999999998,"15min":9.593},"last":0.26000000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:14:59 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.82599999999999996,"15min":0.80300000000000005},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.20999999999999999},"max":{"1min":5.1280000000000001,"5min":7.1280000000000001,"15min":10.403},"last":4.6849999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.68400000000000005,"15min":0.875},"min":{"1min":0.27300000000000002,"5min":0.21199999999999999,"15min":0.20000000000000001},"max":{"1min":5.0800000000000001,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.29899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739283,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6852780,"kb_used_data":5235492,"kb_used_omap":644,"kb_used_meta":1616635,"kb_avail":87519060,"statfs":{"total":96636764160,"available":89619517440,"internally_reserved":0,"allocated":5361143808,"data_stored":8370067447,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1655435170},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.76900000000000002,"15min":0.65900000000000003},"min":{"1min":0.183,"5min":0.183,"15min":0.182},"max":{"1min":3.9409999999999998,"5min":6.4459999999999997,"15min":6.6829999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.4139999999999999,"5min":0.89100000000000001,"15min":0.83399999999999996},"min":{"1min":0.248,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":5.1340000000000003,"5min":6.5,"15min":9.4849999999999994},"last":0.75800000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.165,"15min":1.109},"min":{"1min":0.23000000000000001,"5min":0.215,"15min":0.185},"max":{"1min":3.7469999999999999,"5min":7.601,"15min":12.622999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":1.4650000000000001,"5min":0.92600000000000005,"15min":0.86399999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.20799999999999999},"max":{"1min":5.3769999999999998,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":0.89600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5360513024,"data_stored":8369551219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8790073344,"data_stored":13728115756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3431972864,"data_stored":5360977081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:16:05.512 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:05.512 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:05.513 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:05.513 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:05.513 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:05.514 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:16:05.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:05.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:05.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.510+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:05.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:05.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:05.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:05.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.710+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:05.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:05.722+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:06.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:06.522+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:06.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:06.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:06.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:06.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:08.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:08.549+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:08.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:08.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:08.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:08.561+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:08.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:08.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:08.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:08.601+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:09.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:09.805+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:09.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:09.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:09.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:09.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:09.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:09.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:09.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:09.849+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:09.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:09.861+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:09.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:09.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:12.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.069+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:12.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:12.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:12.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.109+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:12.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.793+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:16:12.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.793+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:16:12.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.793+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:12.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:12.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:12.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:12.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:12.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:12.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.837+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:12.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.837+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:12.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:12.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:13.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:13.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:13.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:13.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:13.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:13.581+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:16.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:16.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:16.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:16.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:16.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:16.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:16.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:16.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:16.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:16.121+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:16.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:16.121+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:16.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:16.129+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:20.158 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:20.153+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:20.169 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:20.161+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:20.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:20.405+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:20.416 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:20.413+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:21.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:21.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:22.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.613+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:22.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:22.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:22.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:22.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:22.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:22.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.669+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:22.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.669+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:22.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:22.677+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:23.910 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:23.905+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:23.923 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:23.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:24.608 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:16:24.605+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:16:24.885 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:24.881+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:24.892 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:24.885+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:25.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:25.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:25.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:25.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:25.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:25.441+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:25.516 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:16:26.117 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:16:26.117 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:16:26.193 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2431,"stamp":"2023-10-14T01:16:24.739301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27929460300,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2192016,"num_read_kb":8537999,"num_write":2154993,"num_write_kb":8619761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80568,"ondisk_log_size":80568,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21538928,"kb_used_data":17237528,"kb_used_omap":2122,"kb_used_meta":4299253,"kb_avail":261576592,"statfs":{"total":289910292480,"available":267854430208,"internally_reserved":0,"allocated":17651228672,"data_stored":27526424075,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2173927,"internal_metadata":4402435097},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,11,5,16,10,7,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25378816,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6471,"num_read_kb":25784,"num_write":6175,"num_write_kb":24700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002703"},"pg_stats":[{"pgid":"2.7","version":"129'252911","reported_seq":503555,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:22.989867+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:16:22.989867+0000","last_peered":"2023-10-14T01:16:22.989867+0000","last_clean":"2023-10-14T01:16:22.989867+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:16:22.989867+0000","last_undegraded":"2023-10-14T01:16:22.989867+0000","last_fullsized":"2023-10-14T01:16:22.989867+0000","mapping_epoch":13,"log_start":"125'242811","ondisk_log_start":"125'242811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3252007095,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":251499,"num_read_kb":991072,"num_write":251855,"num_write_kb":1007405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"129'265579","reported_seq":524932,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:22.962813+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:16:22.962813+0000","last_peered":"2023-10-14T01:16:22.962813+0000","last_clean":"2023-10-14T01:16:22.962813+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:16:22.962813+0000","last_undegraded":"2023-10-14T01:16:22.962813+0000","last_fullsized":"2023-10-14T01:16:22.962813+0000","mapping_epoch":13,"log_start":"126'255508","ondisk_log_start":"126'255508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3277156619,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":261685,"num_read_kb":1009241,"num_write":252427,"num_write_kb":1009307,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"129'271226","reported_seq":539159,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:22.975102+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:16:22.975102+0000","last_peered":"2023-10-14T01:16:22.975102+0000","last_clean":"2023-10-14T01:16:22.975102+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:16:22.975102+0000","last_undegraded":"2023-10-14T01:16:22.975102+0000","last_fullsized":"2023-10-14T01:16:22.975102+0000","mapping_epoch":13,"log_start":"126'261212","ondisk_log_start":"126'261212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":44,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 23s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3445703211,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":268869,"num_read_kb":1055818,"num_write":266000,"num_write_kb":1063879,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"129'286950","reported_seq":564598,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:22.977600+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:16:22.977600+0000","last_peered":"2023-10-14T01:16:22.977600+0000","last_clean":"2023-10-14T01:16:22.977600+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:16:22.977600+0000","last_undegraded":"2023-10-14T01:16:22.977600+0000","last_fullsized":"2023-10-14T01:16:22.977600+0000","mapping_epoch":13,"log_start":"126'276910","ondisk_log_start":"126'276910","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":286,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 95s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3540648367,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":278765,"num_read_kb":1081931,"num_write":273619,"num_write_kb":1094158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"129'276899","reported_seq":553671,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:24.494257+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:16:24.494257+0000","last_peered":"2023-10-14T01:16:24.494257+0000","last_clean":"2023-10-14T01:16:24.494257+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:16:24.494257+0000","last_undegraded":"2023-10-14T01:16:24.494257+0000","last_fullsized":"2023-10-14T01:16:24.494257+0000","mapping_epoch":13,"log_start":"126'266812","ondisk_log_start":"126'266812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3488756183,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":277301,"num_read_kb":1068963,"num_write":269802,"num_write_kb":1078701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"129'290496","reported_seq":578036,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:23.816654+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:16:23.816654+0000","last_peered":"2023-10-14T01:16:23.816654+0000","last_clean":"2023-10-14T01:16:23.816654+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:16:23.816654+0000","last_undegraded":"2023-10-14T01:16:23.816654+0000","last_fullsized":"2023-10-14T01:16:23.816654+0000","mapping_epoch":13,"log_start":"126'280412","ondisk_log_start":"126'280412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3702669732,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":288175,"num_read_kb":1127132,"num_write":284358,"num_write_kb":1137258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"129'287730","reported_seq":570743,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:23.803235+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:16:23.803235+0000","last_peered":"2023-10-14T01:16:23.803235+0000","last_clean":"2023-10-14T01:16:23.803235+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:16:23.803235+0000","last_undegraded":"2023-10-14T01:16:23.803235+0000","last_fullsized":"2023-10-14T01:16:23.803235+0000","mapping_epoch":13,"log_start":"126'277715","ondisk_log_start":"126'277715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3653280097,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":283699,"num_read_kb":1109898,"num_write":281589,"num_write_kb":1126166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"129'292344","reported_seq":573201,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:22.965889+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:16:22.965889+0000","last_peered":"2023-10-14T01:16:22.965889+0000","last_clean":"2023-10-14T01:16:22.965889+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:16:22.965889+0000","last_undegraded":"2023-10-14T01:16:22.965889+0000","last_fullsized":"2023-10-14T01:16:22.965889+0000","mapping_epoch":13,"log_start":"126'282314","ondisk_log_start":"126'282314","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":117,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 42s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3568648628,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":281897,"num_read_kb":1093835,"num_write":275174,"num_write_kb":1100339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":336,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:21.037126+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:15:21.037126+0000","last_peered":"2023-10-14T01:15:21.037126+0000","last_clean":"2023-10-14T01:15:21.037126+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:15:21.037126+0000","last_undegraded":"2023-10-14T01:15:21.037126+0000","last_fullsized":"2023-10-14T01:15:21.037126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27928869932,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2191890,"num_read_kb":8537890,"num_write":2154824,"num_write_kb":8617213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17648693248,"data_stored":27524778072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80441,"ondisk_log_size":80441,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739286,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5160496,"kb_used_data":3364840,"kb_used_omap":351,"kb_used_meta":1795296,"kb_avail":89211344,"statfs":{"total":96636764160,"available":91352416256,"internally_reserved":0,"allocated":3445596160,"data_stored":5373948554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1838383809},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.67400000000000004,"15min":0.79700000000000004},"min":{"1min":0.23499999999999999,"5min":0.193,"15min":0.17299999999999999},"max":{"1min":3.5339999999999998,"5min":6.1120000000000001,"15min":16.474},"last":4.6239999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.84599999999999997,"15min":0.83399999999999996},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":4.3730000000000002,"5min":7.8499999999999996,"15min":11.554},"last":0.25}]},{"osd":1,"last update":"Sat Oct 14 01:15:23 2023","interfaces":[{"interface":"back","average":{"1min":2.1829999999999998,"5min":2.3639999999999999,"15min":1.8169999999999999},"min":{"1min":0.28299999999999997,"5min":0.23400000000000001,"15min":0.20200000000000001},"max":{"1min":10.388,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":2.2290000000000001,"5min":2.4399999999999999,"15min":1.913},"min":{"1min":0.313,"5min":0.27600000000000002,"15min":0.252},"max":{"1min":10.436999999999999,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.46999999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739278,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9433772,"kb_used_data":8617508,"kb_used_omap":1127,"kb_used_meta":815128,"kb_avail":84938068,"statfs":{"total":96636764160,"available":86976581632,"internally_reserved":0,"allocated":8824328192,"data_stored":13762251805,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":834691510},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,4,3,10,5,7,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":2.0350000000000001,"5min":0.84099999999999997,"15min":0.90600000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.184},"max":{"1min":15.132999999999999,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":2.1629999999999998,"5min":0.91200000000000003,"15min":0.749},"min":{"1min":0.252,"5min":0.252,"15min":0.21299999999999999},"max":{"1min":15.173,"5min":15.173,"15min":15.173},"last":0.52600000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":0.94699999999999995,"15min":0.83399999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":4.6849999999999996,"5min":7.1280000000000001,"15min":10.403},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.86699999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20000000000000001},"max":{"1min":4.6870000000000003,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.79600000000000004}]}]},{"osd":0,"up_from":8,"seq":34359739287,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6944660,"kb_used_data":5255180,"kb_used_omap":644,"kb_used_meta":1688827,"kb_avail":87427180,"statfs":{"total":96636764160,"available":89525432320,"internally_reserved":0,"allocated":5381304320,"data_stored":8390223716,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1729359778},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.76900000000000002,"15min":0.65900000000000003},"min":{"1min":0.183,"5min":0.183,"15min":0.182},"max":{"1min":3.9409999999999998,"5min":6.4459999999999997,"15min":6.6829999999999998},"last":0.40500000000000003},{"interface":"front","average":{"1min":1.4139999999999999,"5min":0.89100000000000001,"15min":0.83399999999999996},"min":{"1min":0.248,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":5.1340000000000003,"5min":6.5,"15min":9.4849999999999994},"last":0.30299999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.165,"15min":1.109},"min":{"1min":0.23000000000000001,"5min":0.215,"15min":0.185},"max":{"1min":3.7469999999999999,"5min":7.601,"15min":12.622999999999999},"last":0.22},{"interface":"front","average":{"1min":1.4650000000000001,"5min":0.92600000000000005,"15min":0.86399999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.20799999999999999},"max":{"1min":5.3769999999999998,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":0.432}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5380788224,"data_stored":8389826419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8823279616,"data_stored":13761322028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3444625408,"data_stored":5373629625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:16:26.195 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:26.195 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:26.195 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:26.196 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:26.196 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:26.197 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:16:26.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:26.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:26.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:26.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:26.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:26.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:26.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:26.321+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:26.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:26.341+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:28.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.677+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:28.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.717+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:28.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:28.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:28.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:28.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:28.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.789+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:28.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.789+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:28.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.925+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:28.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:28.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:28.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.929+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:28.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.933+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:28.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:28.965+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:30.732 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:30.729+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:16:30.739 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:30.733+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:16:30.757 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:30.753+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:30.779 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:30.777+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:30.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:30.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:30.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:30.797+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:30.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:30.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:30.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:30.805+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:30.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:30.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:30.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:30.961+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:31.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:31.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:31.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:31.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:31.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:31.001+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:31.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:31.001+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:31.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:31.001+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:33.556 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:33.553+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:33.594 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:33.589+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:36.117 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:36.113+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:36.343 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:36.341+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:36.759 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:36.757+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:36.760 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:36.757+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:36.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:36.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:36.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:36.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:36.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:36.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:36.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:36.913+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:36.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:36.913+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:37.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:37.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:37.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:37.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:37.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:37.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.697+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:37.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.697+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:37.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:37.705+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:38.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:38.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:38.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:38.093+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:38.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:38.881+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:38.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:38.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:38.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:38.917+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:39.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.329+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:39.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:39.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:39.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:39.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:39.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.389+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:39.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.389+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:39.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:39.393+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:40.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.361+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:40.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:40.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:40.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:40.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:40.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.453+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:40.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.453+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:40.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:40.461+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:41.210 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:41.205+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:41.228 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:41.225+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:41.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:41.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:41.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:41.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:41.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:41.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:41.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:41.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:41.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:41.501+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:41.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:41.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:41.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:41.505+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:42.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:42.972+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:42.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:42.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:43.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:43.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:43.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:43.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:43.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:43.264+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:43.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:43.272+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:43.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:43.292+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:43.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:43.316+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:43.851 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:43.840+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:46.198 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:16:47.056 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:16:47.057 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:16:47.090 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2442,"stamp":"2023-10-14T01:16:46.746963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27963047500,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2200474,"num_read_kb":8571687,"num_write":2163164,"num_write_kb":8652445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80557,"ondisk_log_size":80557,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21901596,"kb_used_data":17302084,"kb_used_omap":2122,"kb_used_meta":4597365,"kb_avail":261213924,"statfs":{"total":289910292480,"available":267483058176,"internally_reserved":0,"allocated":17717334016,"data_stored":27592529419,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2173927,"internal_metadata":4707701785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,3,3,8,16,24,20],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16928768,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4294,"num_read_kb":17108,"num_write":4111,"num_write_kb":16444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004107"},"pg_stats":[{"pgid":"2.7","version":"129'253861","reported_seq":505495,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:43.525986+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:16:43.525986+0000","last_peered":"2023-10-14T01:16:43.525986+0000","last_clean":"2023-10-14T01:16:43.525986+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:16:43.525986+0000","last_undegraded":"2023-10-14T01:16:43.525986+0000","last_fullsized":"2023-10-14T01:16:43.525986+0000","mapping_epoch":13,"log_start":"126'243811","ondisk_log_start":"126'243811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3255886007,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":252486,"num_read_kb":995012,"num_write":252805,"num_write_kb":1011205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"129'266612","reported_seq":526970,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:43.526035+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:16:43.526035+0000","last_peered":"2023-10-14T01:16:43.526035+0000","last_clean":"2023-10-14T01:16:43.526035+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:16:43.526035+0000","last_undegraded":"2023-10-14T01:16:43.526035+0000","last_fullsized":"2023-10-14T01:16:43.526035+0000","mapping_epoch":13,"log_start":"126'256608","ondisk_log_start":"126'256608","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3281203467,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":262677,"num_read_kb":1013197,"num_write":253414,"num_write_kb":1013255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"129'272301","reported_seq":541268,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:43.505124+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:16:43.505124+0000","last_peered":"2023-10-14T01:16:43.505124+0000","last_clean":"2023-10-14T01:16:43.505124+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:16:43.505124+0000","last_undegraded":"2023-10-14T01:16:43.505124+0000","last_fullsized":"2023-10-14T01:16:43.505124+0000","mapping_epoch":13,"log_start":"126'262212","ondisk_log_start":"126'262212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":96,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 43s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3449958955,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":269894,"num_read_kb":1059894,"num_write":267026,"num_write_kb":1067983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"129'288090","reported_seq":566865,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:43.517330+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:16:43.517330+0000","last_peered":"2023-10-14T01:16:43.517330+0000","last_clean":"2023-10-14T01:16:43.517330+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:16:43.517330+0000","last_undegraded":"2023-10-14T01:16:43.517330+0000","last_fullsized":"2023-10-14T01:16:43.517330+0000","mapping_epoch":13,"log_start":"126'278010","ondisk_log_start":"126'278010","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":332,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 116s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3545010607,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":279873,"num_read_kb":1086343,"num_write":274684,"num_write_kb":1098418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"129'277868","reported_seq":555715,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:44.732700+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:16:44.732700+0000","last_peered":"2023-10-14T01:16:44.732700+0000","last_clean":"2023-10-14T01:16:44.732700+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:16:44.732700+0000","last_undegraded":"2023-10-14T01:16:44.732700+0000","last_fullsized":"2023-10-14T01:16:44.732700+0000","mapping_epoch":13,"log_start":"126'267812","ondisk_log_start":"126'267812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3492704727,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":278356,"num_read_kb":1073175,"num_write":270756,"num_write_kb":1082517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"129'291586","reported_seq":580254,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:43.908626+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:16:43.908626+0000","last_peered":"2023-10-14T01:16:43.908626+0000","last_clean":"2023-10-14T01:16:43.908626+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:16:43.908626+0000","last_undegraded":"2023-10-14T01:16:43.908626+0000","last_fullsized":"2023-10-14T01:16:43.908626+0000","mapping_epoch":13,"log_start":"126'281512","ondisk_log_start":"126'281512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3707068836,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":289293,"num_read_kb":1131576,"num_write":285428,"num_write_kb":1141538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"129'288820","reported_seq":572928,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:16:43.890799+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:16:43.890799+0000","last_peered":"2023-10-14T01:16:43.890799+0000","last_clean":"2023-10-14T01:16:43.890799+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:16:43.890799+0000","last_undegraded":"2023-10-14T01:16:43.890799+0000","last_fullsized":"2023-10-14T01:16:43.890799+0000","mapping_epoch":13,"log_start":"126'278815","ondisk_log_start":"126'278815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3657683297,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":284790,"num_read_kb":1114242,"num_write":282661,"num_write_kb":1130454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"129'293486","reported_seq":575448,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:16:43.532514+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:16:43.532514+0000","last_peered":"2023-10-14T01:16:43.532514+0000","last_clean":"2023-10-14T01:16:43.532514+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:16:43.532514+0000","last_undegraded":"2023-10-14T01:16:43.532514+0000","last_fullsized":"2023-10-14T01:16:43.532514+0000","mapping_epoch":13,"log_start":"126'283414","ondisk_log_start":"126'283414","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":167,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 62s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3572941236,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":282979,"num_read_kb":1098139,"num_write":276221,"num_write_kb":1104527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":336,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:21.037126+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:15:21.037126+0000","last_peered":"2023-10-14T01:15:21.037126+0000","last_clean":"2023-10-14T01:15:21.037126+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:15:21.037126+0000","last_undegraded":"2023-10-14T01:15:21.037126+0000","last_fullsized":"2023-10-14T01:15:21.037126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27962457132,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2200348,"num_read_kb":8571578,"num_write":2162995,"num_write_kb":8649897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17715425280,"data_stored":27591510104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80430,"ondisk_log_size":80430,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739290,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5245252,"kb_used_data":3377468,"kb_used_omap":351,"kb_used_meta":1867424,"kb_avail":89126588,"statfs":{"total":96636764160,"available":91265626112,"internally_reserved":0,"allocated":3458527232,"data_stored":5386879626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1912242881},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.17299999999999999},"max":{"1min":5.2629999999999999,"5min":6.1120000000000001,"15min":16.474},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.86599999999999999,"15min":0.88},"min":{"1min":0.22800000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.1280000000000001,"5min":6.0990000000000002,"15min":11.554},"last":10.871}]},{"osd":1,"last update":"Sat Oct 14 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":2.4900000000000002,"15min":1.778},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.20200000000000001},"max":{"1min":11.298,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":2.5659999999999998,"15min":1.873},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.252},"max":{"1min":11.336,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.92300000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739282,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9615900,"kb_used_data":8650004,"kb_used_omap":1127,"kb_used_meta":964760,"kb_avail":84755940,"statfs":{"total":96636764160,"available":86790082560,"internally_reserved":0,"allocated":8857604096,"data_stored":13795527709,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":987914678},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,2,0,2,6,9,14],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":2.0350000000000001,"5min":0.84099999999999997,"15min":0.90600000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.184},"max":{"1min":15.132999999999999,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":2.1629999999999998,"5min":0.91200000000000003,"15min":0.749},"min":{"1min":0.252,"5min":0.252,"15min":0.21299999999999999},"max":{"1min":15.173,"5min":15.173,"15min":15.173},"last":0.29699999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":0.94699999999999995,"15min":0.83399999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":4.6849999999999996,"5min":7.1280000000000001,"15min":10.403},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.86699999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20000000000000001},"max":{"1min":4.6870000000000003,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.25900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739291,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7040444,"kb_used_data":5274612,"kb_used_omap":644,"kb_used_meta":1765179,"kb_avail":87331396,"statfs":{"total":96636764160,"available":89427349504,"internally_reserved":0,"allocated":5401202688,"data_stored":8410122084,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1807544226},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,4,4,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.76900000000000002,"15min":0.65900000000000003},"min":{"1min":0.183,"5min":0.183,"15min":0.182},"max":{"1min":3.9409999999999998,"5min":6.4459999999999997,"15min":6.6829999999999998},"last":0.499},{"interface":"front","average":{"1min":1.4139999999999999,"5min":0.89100000000000001,"15min":0.83399999999999996},"min":{"1min":0.248,"5min":0.21199999999999999,"15min":0.191},"max":{"1min":5.1340000000000003,"5min":6.5,"15min":9.4849999999999994},"last":0.30399999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:16:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":1.165,"15min":1.109},"min":{"1min":0.23000000000000001,"5min":0.215,"15min":0.185},"max":{"1min":3.7469999999999999,"5min":7.601,"15min":12.622999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.4650000000000001,"5min":0.92600000000000005,"15min":0.86399999999999999},"min":{"1min":0.26700000000000002,"5min":0.23000000000000001,"15min":0.20799999999999999},"max":{"1min":5.3769999999999998,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":0.26700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5400748032,"data_stored":8409786227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8857030656,"data_stored":13795073068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3457646592,"data_stored":5386650809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:16:47.091 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:47.092 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:47.092 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:47.093 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:47.093 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:16:47.094 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:16:47.565 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:47.560+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:47.580 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:47.576+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:48.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.580+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:48.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:48.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:48.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:48.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:48.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:48.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.680+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:48.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.692+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:48.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:48.696+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:50.806 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:50.800+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:50.811 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:50.808+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:51.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:51.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:51.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:51.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.128+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:51.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.592+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:51.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:51.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:51.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:51.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.656+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:51.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.656+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:51.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.656+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:51.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:51.656+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:52.444 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:52.440+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:16:52.467 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:52.460+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:52.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.676+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:52.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:52.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:52.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.708+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:52.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:52.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:52.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:52.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:52.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.884+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:52.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:52.884+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:55.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.004+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:55.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:55.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:55.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:55.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:55.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:55.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.064+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:16:55.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.064+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:16:55.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.076+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:55.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.120+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:55.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:55.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:16:55.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:16:55.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.180+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:55.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.876+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:55.881 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:55.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:55.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:55.892+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:56.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:56.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:16:56.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:56.392+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:56.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:56.416+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:57.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:57.504+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:57.511 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:57.508+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:16:57.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:57.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:57.526 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:57.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:16:57.551 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:57.548+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:16:57.558 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:16:57.552+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:16:59.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:59.648+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:16:59.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:59.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:16:59.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:59.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:16:59.680 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:16:59.676+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:00.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.764+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:00.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:00.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:00.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:00.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:00.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.812+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:00.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.812+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:00.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:00.820+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:01.774 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:01.768+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:02.874 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:02.864+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:17:02.889 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:02.884+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:17:02.892 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:02.888+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:02.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:02.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:02.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:02.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:02.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:02.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:02.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:02.944+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:02.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:02.944+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:02.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:02.944+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:05.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.460+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:05.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:05.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:05.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:05.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.488+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:05.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.736+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:05.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.760+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:05.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.776+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:05.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:05.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.784+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:05.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.784+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:05.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:05.784+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:06.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:06.888+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:06.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:06.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:06.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:06.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:06.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:06.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:06.932+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:07.095 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:17:07.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:07.204+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:07.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:07.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:07.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:07.320+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:07.888 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:17:07.889 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2452,"stamp":"2023-10-14T01:17:06.750188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":27995848268,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2208665,"num_read_kb":8604347,"num_write":2171160,"num_write_kb":8684429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80568,"ondisk_log_size":80568,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22196044,"kb_used_data":17364976,"kb_used_omap":2122,"kb_used_meta":4828917,"kb_avail":260919476,"statfs":{"total":289910292480,"available":267181543424,"internally_reserved":0,"allocated":17781735424,"data_stored":27656930827,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2173927,"internal_metadata":4944811033},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,2,3,7,11,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17362944,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4398,"num_read_kb":17548,"num_write":4233,"num_write_kb":16932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001974"},"pg_stats":[{"pgid":"2.7","version":"129'254770","reported_seq":507330,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:17:03.979951+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:17:03.979951+0000","last_peered":"2023-10-14T01:17:03.979951+0000","last_clean":"2023-10-14T01:17:03.979951+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:17:03.979951+0000","last_undegraded":"2023-10-14T01:17:03.979951+0000","last_fullsized":"2023-10-14T01:17:03.979951+0000","mapping_epoch":13,"log_start":"126'244711","ondisk_log_start":"126'244711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3259633847,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":253412,"num_read_kb":998704,"num_write":253714,"num_write_kb":1014841,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"129'267604","reported_seq":528940,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:17:04.013660+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:17:04.013660+0000","last_peered":"2023-10-14T01:17:04.013660+0000","last_clean":"2023-10-14T01:17:04.013660+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:17:04.013660+0000","last_undegraded":"2023-10-14T01:17:04.013660+0000","last_fullsized":"2023-10-14T01:17:04.013660+0000","mapping_epoch":13,"log_start":"126'257508","ondisk_log_start":"126'257508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3285094667,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":263646,"num_read_kb":1017057,"num_write":254367,"num_write_kb":1017067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"129'273338","reported_seq":543326,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:03.919065+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:17:03.919065+0000","last_peered":"2023-10-14T01:17:03.919065+0000","last_clean":"2023-10-14T01:17:03.919065+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:17:03.919065+0000","last_undegraded":"2023-10-14T01:17:03.919065+0000","last_fullsized":"2023-10-14T01:17:03.919065+0000","mapping_epoch":13,"log_start":"127'263312","ondisk_log_start":"127'263312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":169,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 64s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3453973035,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":270898,"num_read_kb":1063894,"num_write":268002,"num_write_kb":1071887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"129'289177","reported_seq":568996,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:03.945723+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:17:03.945723+0000","last_peered":"2023-10-14T01:17:03.945723+0000","last_clean":"2023-10-14T01:17:03.945723+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:17:03.945723+0000","last_undegraded":"2023-10-14T01:17:03.945723+0000","last_fullsized":"2023-10-14T01:17:03.945723+0000","mapping_epoch":13,"log_start":"127'279110","ondisk_log_start":"127'279110","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":407,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 136s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3549110703,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":280894,"num_read_kb":1090411,"num_write":275681,"num_write_kb":1102406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"129'278841","reported_seq":557690,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:17:04.817850+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:17:04.817850+0000","last_peered":"2023-10-14T01:17:04.817850+0000","last_clean":"2023-10-14T01:17:04.817850+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:17:04.817850+0000","last_undegraded":"2023-10-14T01:17:04.817850+0000","last_fullsized":"2023-10-14T01:17:04.817850+0000","mapping_epoch":13,"log_start":"127'268812","ondisk_log_start":"127'268812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3496669655,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":279337,"num_read_kb":1077091,"num_write":271724,"num_write_kb":1086389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"129'292745","reported_seq":582557,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:17:04.191448+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:17:04.191448+0000","last_peered":"2023-10-14T01:17:04.191448+0000","last_clean":"2023-10-14T01:17:04.191448+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:17:04.191448+0000","last_undegraded":"2023-10-14T01:17:04.191448+0000","last_fullsized":"2023-10-14T01:17:04.191448+0000","mapping_epoch":13,"log_start":"127'282712","ondisk_log_start":"127'282712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3711738276,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":290433,"num_read_kb":1136116,"num_write":286571,"num_write_kb":1146110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"129'289877","reported_seq":575057,"reported_epoch":129,"state":"active+clean","last_fresh":"2023-10-14T01:17:04.176798+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:17:04.176798+0000","last_peered":"2023-10-14T01:17:04.176798+0000","last_clean":"2023-10-14T01:17:04.176798+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:17:04.176798+0000","last_undegraded":"2023-10-14T01:17:04.176798+0000","last_fullsized":"2023-10-14T01:17:04.176798+0000","mapping_epoch":13,"log_start":"127'279815","ondisk_log_start":"127'279815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3661930849,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":285855,"num_read_kb":1118498,"num_write":283700,"num_write_kb":1134610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"129'294583","reported_seq":577656,"reported_epoch":129,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:03.914692+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:17:03.914692+0000","last_peered":"2023-10-14T01:17:03.914692+0000","last_clean":"2023-10-14T01:17:03.914692+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:17:03.914692+0000","last_undegraded":"2023-10-14T01:17:03.914692+0000","last_fullsized":"2023-10-14T01:17:03.914692+0000","mapping_epoch":13,"log_start":"127'284514","ondisk_log_start":"127'284514","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":216,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 83s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3577106868,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":284064,"num_read_kb":1102467,"num_write":277232,"num_write_kb":1108571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":336,"reported_epoch":128,"state":"active+clean","last_fresh":"2023-10-14T01:15:21.037126+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:15:21.037126+0000","last_peered":"2023-10-14T01:15:21.037126+0000","last_clean":"2023-10-14T01:15:21.037126+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:15:21.037126+0000","last_undegraded":"2023-10-14T01:15:21.037126+0000","last_fullsized":"2023-10-14T01:15:21.037126+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":27995257900,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2208539,"num_read_kb":8604238,"num_write":2170991,"num_write_kb":8681881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17780985856,"data_stored":27657070680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80441,"ondisk_log_size":80441,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739294,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5289532,"kb_used_data":3389876,"kb_used_omap":351,"kb_used_meta":1899296,"kb_avail":89082308,"statfs":{"total":96636764160,"available":91220283392,"internally_reserved":0,"allocated":3471233024,"data_stored":5399585418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":1944879809},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.17299999999999999},"max":{"1min":5.2629999999999999,"5min":6.1120000000000001,"15min":16.474},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.86599999999999999,"15min":0.88},"min":{"1min":0.22800000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.1280000000000001,"5min":6.0990000000000002,"15min":11.554},"last":0.22800000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":2.4900000000000002,"15min":1.778},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.20200000000000001},"max":{"1min":11.298,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.38600000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":2.5659999999999998,"15min":1.873},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.252},"max":{"1min":11.336,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.46300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739286,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9775756,"kb_used_data":8681344,"kb_used_omap":1127,"kb_used_meta":1093272,"kb_avail":84596084,"statfs":{"total":96636764160,"available":86626390016,"internally_reserved":0,"allocated":8889696256,"data_stored":13827619869,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":1119510966},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,6,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":2.0350000000000001,"5min":0.84099999999999997,"15min":0.90600000000000003},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.184},"max":{"1min":15.132999999999999,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":2.1629999999999998,"5min":0.91200000000000003,"15min":0.749},"min":{"1min":0.252,"5min":0.252,"15min":0.21299999999999999},"max":{"1min":15.173,"5min":15.173,"15min":15.173},"last":0.61899999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:16:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":0.94699999999999995,"15min":0.83399999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.20999999999999999},"max":{"1min":4.6849999999999996,"5min":7.1280000000000001,"15min":10.403},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.86699999999999999},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20000000000000001},"max":{"1min":4.6870000000000003,"5min":7.1989999999999998,"15min":9.2200000000000006},"last":0.26900000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739295,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7130756,"kb_used_data":5293756,"kb_used_omap":644,"kb_used_meta":1836347,"kb_avail":87241084,"statfs":{"total":96636764160,"available":89334870016,"internally_reserved":0,"allocated":5420806144,"data_stored":8429725540,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1880420258},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,1,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6759999999999999,"5min":0.98699999999999999,"15min":0.70299999999999996},"min":{"1min":0.20000000000000001,"5min":0.183,"15min":0.182},"max":{"1min":19.274000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.0980000000000001,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.217,"15min":0.191},"max":{"1min":19.376000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.32100000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.141,"15min":1.103},"min":{"1min":0.216,"5min":0.215,"15min":0.185},"max":{"1min":9.3309999999999995,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.442},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.91500000000000004,"15min":0.82699999999999996},"min":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.20999999999999999},"max":{"1min":4.2240000000000002,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":0.28000000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5420236800,"data_stored":8429274995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8890007552,"data_stored":13828049964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3470741504,"data_stored":5399745721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:17:07.961 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:07.961 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:07.961 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:07.962 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:07.962 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:07.963 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:17:11.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.124+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:11.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:11.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:11.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.144+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:11.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:11.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:11.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.164+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:11.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.164+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:11.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.168+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:11.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.820+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:11.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:11.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:11.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:11.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.928+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:11.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:11.928+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:12.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:12.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:12.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:12.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.372+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:12.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.372+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:12.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.372+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:12.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.584+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:12.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.636+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:12.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:12.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:12.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.652+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:12.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.652+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:12.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:12.656+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:13.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:13.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:13.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:13.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:13.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:13.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:13.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:13.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:13.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:13.384+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:14.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.404+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:17:14.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.404+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:17:14.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.404+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:14.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.412+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:14.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:14.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:14.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:14.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:14.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.468+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:14.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.468+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:14.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:14.468+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:15.197 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:15.196+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:17:15.214 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:15.212+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:16.760 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:16.760+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:17.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:17.612+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:17.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:17.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:17.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:17.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:17.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:17.732+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:19.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:19.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:19.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:19.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:19.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:19.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:19.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:19.023+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:19.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:19.023+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:20.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.647+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:20.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:20.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:20.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:20.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:20.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:20.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.747+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:20.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.747+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:20.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:20.751+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:21.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:21.331+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:21.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:21.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:21.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:21.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:21.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:21.387+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:23.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:23.915+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:23.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:23.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:23.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:23.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:23.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:23.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:23.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:23.963+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:23.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:23.963+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:23.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:23.963+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:25.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.439+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:25.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:25.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:25.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:25.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:25.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:25.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.479+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:25.848 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:25.847+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:17:25.849 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:25.847+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:25.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:25.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:25.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:25.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.975+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:25.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.975+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:25.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.975+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:25.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:25.975+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:27.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:27.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:27.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:27.911+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:27.936 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:27.935+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:27.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:17:28.898 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:17:28.902 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:17:28.958 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2464,"stamp":"2023-10-14T01:17:28.753679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28028743244,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2216492,"num_read_kb":8635523,"num_write":2179106,"num_write_kb":8716213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80455,"ondisk_log_size":80455,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22455432,"kb_used_data":17429232,"kb_used_omap":2122,"kb_used_meta":5024053,"kb_avail":260660088,"statfs":{"total":289910292480,"available":266915930112,"internally_reserved":0,"allocated":17847533568,"data_stored":27722716113,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2173927,"internal_metadata":5144630297},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,5,7,15,6,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16633856,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3876,"num_read_kb":15444,"num_write":4016,"num_write_kb":16064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.173789"},"pg_stats":[{"pgid":"2.7","version":"130'255662","reported_seq":509118,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:24.520268+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:17:24.520268+0000","last_peered":"2023-10-14T01:17:24.520268+0000","last_clean":"2023-10-14T01:17:24.520268+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:17:24.520268+0000","last_undegraded":"2023-10-14T01:17:24.520268+0000","last_fullsized":"2023-10-14T01:17:24.520268+0000","mapping_epoch":13,"log_start":"126'245611","ondisk_log_start":"126'245611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3263295671,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":254307,"num_read_kb":1002276,"num_write":254606,"num_write_kb":1018409,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"130'268516","reported_seq":530798,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:24.509294+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:17:24.509294+0000","last_peered":"2023-10-14T01:17:24.509294+0000","last_clean":"2023-10-14T01:17:24.509294+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:17:24.509294+0000","last_undegraded":"2023-10-14T01:17:24.509294+0000","last_fullsized":"2023-10-14T01:17:24.509294+0000","mapping_epoch":13,"log_start":"127'258508","ondisk_log_start":"127'258508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3288699147,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":264577,"num_read_kb":1020757,"num_write":255234,"num_write_kb":1020535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"130'274359","reported_seq":545308,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:24.513998+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:17:24.513998+0000","last_peered":"2023-10-14T01:17:24.513998+0000","last_clean":"2023-10-14T01:17:24.513998+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:17:24.513998+0000","last_undegraded":"2023-10-14T01:17:24.513998+0000","last_fullsized":"2023-10-14T01:17:24.513998+0000","mapping_epoch":13,"log_start":"127'264312","ondisk_log_start":"127'264312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":215,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 84s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3458011691,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":271854,"num_read_kb":1067694,"num_write":268974,"num_write_kb":1075775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"130'290300","reported_seq":571161,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:24.524134+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:17:24.524134+0000","last_peered":"2023-10-14T01:17:24.524134+0000","last_clean":"2023-10-14T01:17:24.524134+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:17:24.524134+0000","last_undegraded":"2023-10-14T01:17:24.524134+0000","last_fullsized":"2023-10-14T01:17:24.524134+0000","mapping_epoch":13,"log_start":"127'280211","ondisk_log_start":"127'280211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":476,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 157s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3553345967,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":281916,"num_read_kb":1094491,"num_write":276709,"num_write_kb":1106518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"130'279852","reported_seq":559695,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:25.024613+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:17:25.024613+0000","last_peered":"2023-10-14T01:17:25.024613+0000","last_clean":"2023-10-14T01:17:25.024613+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:17:25.024613+0000","last_undegraded":"2023-10-14T01:17:25.024613+0000","last_fullsized":"2023-10-14T01:17:25.024613+0000","mapping_epoch":13,"log_start":"127'269812","ondisk_log_start":"127'269812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3500818903,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":280314,"num_read_kb":1080991,"num_write":272710,"num_write_kb":1090333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"130'293891","reported_seq":584717,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:24.411417+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:17:24.411417+0000","last_peered":"2023-10-14T01:17:24.411417+0000","last_clean":"2023-10-14T01:17:24.411417+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:17:24.411417+0000","last_undegraded":"2023-10-14T01:17:24.411417+0000","last_fullsized":"2023-10-14T01:17:24.411417+0000","mapping_epoch":13,"log_start":"127'283812","ondisk_log_start":"127'283812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3716420004,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":291437,"num_read_kb":1140112,"num_write":287703,"num_write_kb":1150638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"130'290922","reported_seq":577105,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:24.387770+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:17:24.387770+0000","last_peered":"2023-10-14T01:17:24.387770+0000","last_clean":"2023-10-14T01:17:24.387770+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:17:24.387770+0000","last_undegraded":"2023-10-14T01:17:24.387770+0000","last_fullsized":"2023-10-14T01:17:24.387770+0000","mapping_epoch":13,"log_start":"127'280915","ondisk_log_start":"127'280915","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3666174305,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":286847,"num_read_kb":1122450,"num_write":284731,"num_write_kb":1138734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"130'295722","reported_seq":579864,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:24.488869+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:17:24.488869+0000","last_peered":"2023-10-14T01:17:24.488869+0000","last_clean":"2023-10-14T01:17:24.488869+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:17:24.488869+0000","last_undegraded":"2023-10-14T01:17:24.488869+0000","last_fullsized":"2023-10-14T01:17:24.488869+0000","mapping_epoch":13,"log_start":"127'285715","ondisk_log_start":"127'285715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":276,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 103s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3581387188,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":285114,"num_read_kb":1106643,"num_write":278270,"num_write_kb":1112723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":338,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:18.595036+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:17:18.595036+0000","last_peered":"2023-10-14T01:17:18.595036+0000","last_clean":"2023-10-14T01:17:18.595036+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:17:18.595036+0000","last_undegraded":"2023-10-14T01:17:18.595036+0000","last_fullsized":"2023-10-14T01:17:18.595036+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28028152876,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2216366,"num_read_kb":8635414,"num_write":2178937,"num_write_kb":8713665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17845866496,"data_stored":27721951320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80328,"ondisk_log_size":80328,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739298,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5370360,"kb_used_data":3402672,"kb_used_omap":351,"kb_used_meta":1967328,"kb_avail":89001480,"statfs":{"total":96636764160,"available":91137515520,"internally_reserved":0,"allocated":3484336128,"data_stored":5412684236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":2014544577},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":0.84899999999999998,"15min":0.86299999999999999},"min":{"1min":0.21299999999999999,"5min":0.20200000000000001,"15min":0.17299999999999999},"max":{"1min":5.2629999999999999,"5min":6.1120000000000001,"15min":16.474},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.86599999999999999,"15min":0.88},"min":{"1min":0.22800000000000001,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.1280000000000001,"5min":6.0990000000000002,"15min":11.554},"last":0.30499999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:16:25 2023","interfaces":[{"interface":"back","average":{"1min":1.679,"5min":2.4900000000000002,"15min":1.778},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.20200000000000001},"max":{"1min":11.298,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.32600000000000001},{"interface":"front","average":{"1min":1.7210000000000001,"5min":2.5659999999999998,"15min":1.873},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.252},"max":{"1min":11.336,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.42299999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739290,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9859520,"kb_used_data":8714360,"kb_used_omap":1127,"kb_used_meta":1144024,"kb_avail":84512320,"statfs":{"total":96636764160,"available":86540615680,"internally_reserved":0,"allocated":8923504640,"data_stored":13861423967,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":1171481014},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,5,8,2,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.84499999999999997,"15min":0.87},"min":{"1min":0.191,"5min":0.191,"15min":0.184},"max":{"1min":4.1180000000000003,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.92400000000000004,"15min":0.74299999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.222},"max":{"1min":3.004,"5min":15.173,"15min":15.173},"last":0.372}]},{"osd":2,"last update":"Sat Oct 14 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.995,"15min":0.83799999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.20999999999999999},"max":{"1min":5.9800000000000004,"5min":7.1280000000000001,"15min":10.403},"last":2.4220000000000002},{"interface":"front","average":{"1min":1.206,"5min":0.76500000000000001,"15min":0.877},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":7.6299999999999999,"5min":7.6299999999999999,"15min":8.9179999999999993},"last":0.24199999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739299,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7225552,"kb_used_data":5312200,"kb_used_omap":644,"kb_used_meta":1912699,"kb_avail":87146288,"statfs":{"total":96636764160,"available":89237798912,"internally_reserved":0,"allocated":5439692800,"data_stored":8448607910,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":1958604706},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6759999999999999,"5min":0.98699999999999999,"15min":0.70299999999999996},"min":{"1min":0.20000000000000001,"5min":0.183,"15min":0.182},"max":{"1min":19.274000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.0980000000000001,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.217,"15min":0.191},"max":{"1min":19.376000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.48999999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.141,"15min":1.103},"min":{"1min":0.216,"5min":0.215,"15min":0.185},"max":{"1min":9.3309999999999995,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.91500000000000004,"15min":0.82699999999999996},"min":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.20999999999999999},"max":{"1min":4.2240000000000002,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":0.52600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5439401984,"data_stored":8448440179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8922722304,"data_stored":13860764716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3483742208,"data_stored":5412746425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:17:28.959 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:28.960 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:28.960 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:28.961 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:28.961 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:28.961 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:17:32.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.231+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:32.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.279+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:32.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:32.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.411+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:32.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.455+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:17:32.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.455+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:17:32.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.455+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:32.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:32.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:32.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:32.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.499+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:32.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.615+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:32.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:32.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:32.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:32.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:32.655+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:33.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.463+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:33.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:33.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:33.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:33.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.539+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:33.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:33.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:33.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:33.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:33.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.919+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:33.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:33.935+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:34.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.883+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:34.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:34.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:34.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:34.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:34.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.907+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:34.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.907+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:34.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:34.907+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:35.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.215+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:35.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:35.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:35.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:35.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:35.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.267+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:35.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.267+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:35.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:35.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:35.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:35.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:35.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:35.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.635+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:35.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.635+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:35.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:35.655+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:37.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:37.611+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:37.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:37.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:37.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:37.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:37.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:37.635+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:38.562 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:38.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:17:38.562 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:38.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:17:38.563 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:38.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:40.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:40.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:40.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:40.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:40.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:40.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:40.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:40.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:40.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:40.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:41.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:41.615+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:41.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:41.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:41.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:41.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:41.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:41.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:41.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:41.691+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:41.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:41.695+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:42.017 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:42.015+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:42.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.155+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:42.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.155+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:42.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:42.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:42.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:42.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:42.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.215+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:42.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.215+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:42.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:42.219+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:43.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:43.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:43.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.831+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:43.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.831+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:43.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.831+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:43.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.831+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:43.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.831+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:43.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:43.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:43.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:43.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.851+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:43.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.851+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:43.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.851+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:43.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:43.851+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:44.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:44.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:44.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:44.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:44.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:44.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.147+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:44.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.147+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:44.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:44.163+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:45.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:45.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.427+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:45.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:45.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:45.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:45.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:45.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.951+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:45.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.951+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:45.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:45.951+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:46.130 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:46.127+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:17:46.163 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:46.159+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:46.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.527+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:46.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:46.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:46.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:46.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:46.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:46.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.595+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:46.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.595+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:46.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:46.595+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:48.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:17:49.575 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:17:49.575 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:17:49.620 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2474,"stamp":"2023-10-14T01:17:48.756918+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28062576204,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2224693,"num_read_kb":8668223,"num_write":2187276,"num_write_kb":8748893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80439,"ondisk_log_size":80439,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22789920,"kb_used_data":17493956,"kb_used_omap":2122,"kb_used_meta":5293813,"kb_avail":260325600,"statfs":{"total":289910292480,"available":266573414400,"internally_reserved":0,"allocated":17913810944,"data_stored":27788993489,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2173927,"internal_metadata":5420864537},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,8,14,13,3,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17178624,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4008,"num_read_kb":15976,"num_write":4125,"num_write_kb":16500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001908"},"pg_stats":[{"pgid":"2.7","version":"130'256618","reported_seq":511023,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:45.079601+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:17:45.079601+0000","last_peered":"2023-10-14T01:17:45.079601+0000","last_clean":"2023-10-14T01:17:45.079601+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:17:45.079601+0000","last_undegraded":"2023-10-14T01:17:45.079601+0000","last_fullsized":"2023-10-14T01:17:45.079601+0000","mapping_epoch":13,"log_start":"127'246611","ondisk_log_start":"127'246611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3267305655,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":255256,"num_read_kb":1006060,"num_write":255562,"num_write_kb":1022233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"130'269539","reported_seq":532802,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:45.056585+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:17:45.056585+0000","last_peered":"2023-10-14T01:17:45.056585+0000","last_clean":"2023-10-14T01:17:45.056585+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:17:45.056585+0000","last_undegraded":"2023-10-14T01:17:45.056585+0000","last_fullsized":"2023-10-14T01:17:45.056585+0000","mapping_epoch":13,"log_start":"127'259508","ondisk_log_start":"127'259508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3292692747,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":265547,"num_read_kb":1024629,"num_write":256202,"num_write_kb":1024407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"130'275443","reported_seq":547405,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:44.991668+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:17:44.991668+0000","last_peered":"2023-10-14T01:17:44.991668+0000","last_clean":"2023-10-14T01:17:44.991668+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:17:44.991668+0000","last_undegraded":"2023-10-14T01:17:44.991668+0000","last_fullsized":"2023-10-14T01:17:44.991668+0000","mapping_epoch":13,"log_start":"127'265412","ondisk_log_start":"127'265412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":261,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 105s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3462308395,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":272852,"num_read_kb":1071674,"num_write":270010,"num_write_kb":1079919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"130'291427","reported_seq":573346,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:45.016611+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:17:45.016611+0000","last_peered":"2023-10-14T01:17:45.016611+0000","last_clean":"2023-10-14T01:17:45.016611+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:17:45.016611+0000","last_undegraded":"2023-10-14T01:17:45.016611+0000","last_fullsized":"2023-10-14T01:17:45.016611+0000","mapping_epoch":13,"log_start":"127'281411","ondisk_log_start":"127'281411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":534,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 177s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3557577135,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":282955,"num_read_kb":1098635,"num_write":277744,"num_write_kb":1110658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"130'280853","reported_seq":561743,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:45.156016+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:17:45.156016+0000","last_peered":"2023-10-14T01:17:45.156016+0000","last_clean":"2023-10-14T01:17:45.156016+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:17:45.156016+0000","last_undegraded":"2023-10-14T01:17:45.156016+0000","last_fullsized":"2023-10-14T01:17:45.156016+0000","mapping_epoch":13,"log_start":"127'270812","ondisk_log_start":"127'270812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3504984535,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":281344,"num_read_kb":1085095,"num_write":273711,"num_write_kb":1094337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"130'294981","reported_seq":586906,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:44.606203+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:17:44.606203+0000","last_peered":"2023-10-14T01:17:44.606203+0000","last_clean":"2023-10-14T01:17:44.606203+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:17:44.606203+0000","last_undegraded":"2023-10-14T01:17:44.606203+0000","last_fullsized":"2023-10-14T01:17:44.606203+0000","mapping_epoch":13,"log_start":"127'284912","ondisk_log_start":"127'284912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3720925604,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":292526,"num_read_kb":1144444,"num_write":288783,"num_write_kb":1154958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"130'291963","reported_seq":579219,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:44.560890+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:17:44.560890+0000","last_peered":"2023-10-14T01:17:44.560890+0000","last_clean":"2023-10-14T01:17:44.560890+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:17:44.560890+0000","last_undegraded":"2023-10-14T01:17:44.560890+0000","last_fullsized":"2023-10-14T01:17:44.560890+0000","mapping_epoch":13,"log_start":"127'281915","ondisk_log_start":"127'281915","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3670450529,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":287912,"num_read_kb":1126706,"num_write":285762,"num_write_kb":1142858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"130'296884","reported_seq":582116,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:17:44.983882+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:17:44.983882+0000","last_peered":"2023-10-14T01:17:44.983882+0000","last_clean":"2023-10-14T01:17:44.983882+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:17:44.983882+0000","last_undegraded":"2023-10-14T01:17:44.983882+0000","last_fullsized":"2023-10-14T01:17:44.983882+0000","mapping_epoch":13,"log_start":"127'286815","ondisk_log_start":"127'286815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":321,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 124s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3585741236,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":286175,"num_read_kb":1110871,"num_write":279333,"num_write_kb":1116975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":338,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:18.595036+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:17:18.595036+0000","last_peered":"2023-10-14T01:17:18.595036+0000","last_clean":"2023-10-14T01:17:18.595036+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:17:18.595036+0000","last_undegraded":"2023-10-14T01:17:18.595036+0000","last_fullsized":"2023-10-14T01:17:18.595036+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28061985836,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2224567,"num_read_kb":8668114,"num_write":2187107,"num_write_kb":8746345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17912709120,"data_stored":27788793944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80312,"ondisk_log_size":80312,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739302,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5469548,"kb_used_data":3415840,"kb_used_omap":351,"kb_used_meta":2053344,"kb_avail":88902292,"statfs":{"total":96636764160,"available":91035947008,"internally_reserved":0,"allocated":3497820160,"data_stored":5426168268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":359743,"internal_metadata":2102624961},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,4,3,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.79300000000000004,"15min":0.83399999999999996},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.17299999999999999},"max":{"1min":3.964,"5min":6.1120000000000001,"15min":16.474},"last":0.28499999999999998},{"interface":"front","average":{"1min":1.488,"5min":0.99099999999999999,"15min":0.92700000000000005},"min":{"1min":0.219,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":10.871,"5min":10.871,"15min":11.554},"last":0.23200000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":1.9039999999999999,"5min":2.5409999999999999,"15min":1.821},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.20200000000000001},"max":{"1min":8.8390000000000004,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.253},{"interface":"front","average":{"1min":2.012,"5min":2.601,"15min":1.9199999999999999},"min":{"1min":0.39100000000000001,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":8.8819999999999997,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.32100000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739294,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10003132,"kb_used_data":8746420,"kb_used_omap":1127,"kb_used_meta":1255576,"kb_avail":84368708,"statfs":{"total":96636764160,"available":86393556992,"internally_reserved":0,"allocated":8956334080,"data_stored":13894253407,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":1285710262},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.84499999999999997,"15min":0.87},"min":{"1min":0.191,"5min":0.191,"15min":0.184},"max":{"1min":4.1180000000000003,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.92400000000000004,"15min":0.74299999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.222},"max":{"1min":3.004,"5min":15.173,"15min":15.173},"last":0.29599999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.995,"15min":0.83799999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.20999999999999999},"max":{"1min":5.9800000000000004,"5min":7.1280000000000001,"15min":10.403},"last":0.318},{"interface":"front","average":{"1min":1.206,"5min":0.76500000000000001,"15min":0.877},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":7.6299999999999999,"5min":7.6299999999999999,"15min":8.9179999999999993},"last":0.23599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739303,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7317240,"kb_used_data":5331696,"kb_used_omap":644,"kb_used_meta":1984891,"kb_avail":87054600,"statfs":{"total":96636764160,"available":89143910400,"internally_reserved":0,"allocated":5459656704,"data_stored":8468571814,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":659550,"internal_metadata":2032529314},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6759999999999999,"5min":0.98699999999999999,"15min":0.70299999999999996},"min":{"1min":0.20000000000000001,"5min":0.183,"15min":0.182},"max":{"1min":19.274000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"last":2.6200000000000001},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.0980000000000001,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.217,"15min":0.191},"max":{"1min":19.376000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":2.5920000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.141,"15min":1.103},"min":{"1min":0.216,"5min":0.215,"15min":0.185},"max":{"1min":9.3309999999999995,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.91500000000000004,"15min":0.82699999999999996},"min":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.20999999999999999},"max":{"1min":4.2240000000000002,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":2.4710000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5459550208,"data_stored":8468588403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8956452864,"data_stored":13894495276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3496706048,"data_stored":5425710265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:17:49.621 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:49.622 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:49.622 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:49.622 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:49.623 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:17:49.623 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:17:52.174 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:52.171+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:52.594 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:52.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:55.426 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.422+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:17:55.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.422+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:17:55.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.454+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:55.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:55.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:55.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:55.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:55.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:55.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.510+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:55.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.510+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:55.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:55.522+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:56.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:56.346+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:56.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:56.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:56.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:56.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:56.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:56.398+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:56.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:56.398+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:56.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:56.398+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:56.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:56.398+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:56.589 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:56.586+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:17:56.614 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:56.610+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:57.669 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:57.666+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:17:57.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.762+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:57.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:17:57.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:17:57.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:57.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:17:57.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:17:57.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.810+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:17:57.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.810+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:17:57.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:57.810+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:58.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:58.050+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:17:58.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:58.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:17:58.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:17:58.106+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:17:59.951 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:59.946+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:17:59.964 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:59.962+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:17:59.964 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:17:59.962+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:00.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:00.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:00.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:00.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:00.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:00.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:00.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:00.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:00.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:00.266+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:00.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:00.266+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:00.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:00.266+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:04.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.190+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:04.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:04.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:04.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:04.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:04.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.242+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:04.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.242+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:04.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.242+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:04.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:04.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:05.577 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:05.574+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:05.578 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:05.574+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:05.712 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:05.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:05.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:05.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:05.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:05.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:05.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:05.706+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:05.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:05.754+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:07.059 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:07.054+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:07.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:07.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:07.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:07.590+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:07.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:07.638+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:08.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.642+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:08.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:08.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:08.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:08.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:08.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.738+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:08.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.738+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:08.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.738+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:08.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.738+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:08.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.746+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:08.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.766+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:08.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:08.766+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:09.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:09.578+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:09.625 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:18:09.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:09.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:09.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:09.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:09.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:09.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:09.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:09.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:09.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:09.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:09.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:09.678+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:10.496 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:18:10.496 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:18:10.553 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2484,"stamp":"2023-10-14T01:18:08.760637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28095995468,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2232865,"num_read_kb":8700835,"num_write":2195332,"num_write_kb":8781117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80484,"ondisk_log_size":80484,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21924856,"kb_used_data":17559128,"kb_used_omap":2257,"kb_used_meta":4363438,"kb_avail":261190664,"statfs":{"total":289910292480,"available":267459239936,"internally_reserved":0,"allocated":17980547072,"data_stored":27855729617,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2312180,"internal_metadata":4468160524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,6,9,14,15,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16793600,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3938,"num_read_kb":15724,"num_write":4036,"num_write_kb":16144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001921"},"pg_stats":[{"pgid":"2.7","version":"130'257553","reported_seq":512955,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:05.636615+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:18:05.636615+0000","last_peered":"2023-10-14T01:18:05.636615+0000","last_clean":"2023-10-14T01:18:05.636615+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:18:05.636615+0000","last_undegraded":"2023-10-14T01:18:05.636615+0000","last_fullsized":"2023-10-14T01:18:05.636615+0000","mapping_epoch":13,"log_start":"127'247511","ondisk_log_start":"127'247511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3271192759,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":256253,"num_read_kb":1010028,"num_write":256497,"num_write_kb":1025973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"130'270529","reported_seq":534742,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:05.634803+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:18:05.634803+0000","last_peered":"2023-10-14T01:18:05.634803+0000","last_clean":"2023-10-14T01:18:05.634803+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:18:05.634803+0000","last_undegraded":"2023-10-14T01:18:05.634803+0000","last_fullsized":"2023-10-14T01:18:05.634803+0000","mapping_epoch":13,"log_start":"127'260508","ondisk_log_start":"127'260508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3296637195,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":266485,"num_read_kb":1028373,"num_write":257162,"num_write_kb":1028247,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"130'276480","reported_seq":549440,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:05.605791+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:18:05.605791+0000","last_peered":"2023-10-14T01:18:05.605791+0000","last_clean":"2023-10-14T01:18:05.605791+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:18:05.605791+0000","last_undegraded":"2023-10-14T01:18:05.605791+0000","last_fullsized":"2023-10-14T01:18:05.605791+0000","mapping_epoch":13,"log_start":"127'266412","ondisk_log_start":"127'266412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":325,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 125s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3466420779,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":273831,"num_read_kb":1075586,"num_write":270993,"num_write_kb":1083851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"130'292519","reported_seq":575514,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:05.612082+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:18:05.612082+0000","last_peered":"2023-10-14T01:18:05.612082+0000","last_clean":"2023-10-14T01:18:05.612082+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:18:05.612082+0000","last_undegraded":"2023-10-14T01:18:05.612082+0000","last_fullsized":"2023-10-14T01:18:05.612082+0000","mapping_epoch":13,"log_start":"128'282511","ondisk_log_start":"128'282511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":605,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 198s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3561820591,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":284013,"num_read_kb":1102863,"num_write":278760,"num_write_kb":1114722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"130'281897","reported_seq":563811,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:05.304029+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:18:05.304029+0000","last_peered":"2023-10-14T01:18:05.304029+0000","last_clean":"2023-10-14T01:18:05.304029+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:18:05.304029+0000","last_undegraded":"2023-10-14T01:18:05.304029+0000","last_fullsized":"2023-10-14T01:18:05.304029+0000","mapping_epoch":13,"log_start":"127'271812","ondisk_log_start":"127'271812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3509215703,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":282350,"num_read_kb":1089115,"num_write":274740,"num_write_kb":1098453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"130'296044","reported_seq":589061,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:04.851143+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:18:04.851143+0000","last_peered":"2023-10-14T01:18:04.851143+0000","last_clean":"2023-10-14T01:18:04.851143+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:18:04.851143+0000","last_undegraded":"2023-10-14T01:18:04.851143+0000","last_fullsized":"2023-10-14T01:18:04.851143+0000","mapping_epoch":13,"log_start":"128'286012","ondisk_log_start":"128'286012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3725238692,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":293606,"num_read_kb":1148764,"num_write":289830,"num_write_kb":1159146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"130'293017","reported_seq":581364,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:04.836736+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:18:04.836736+0000","last_peered":"2023-10-14T01:18:04.836736+0000","last_clean":"2023-10-14T01:18:04.836736+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:18:04.836736+0000","last_undegraded":"2023-10-14T01:18:04.836736+0000","last_fullsized":"2023-10-14T01:18:04.836736+0000","mapping_epoch":13,"log_start":"128'283015","ondisk_log_start":"128'283015","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3674812769,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":288996,"num_read_kb":1131030,"num_write":286800,"num_write_kb":1147010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"130'298015","reported_seq":584302,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:05.582364+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:18:05.582364+0000","last_peered":"2023-10-14T01:18:05.582364+0000","last_clean":"2023-10-14T01:18:05.582364+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:18:05.582364+0000","last_undegraded":"2023-10-14T01:18:05.582364+0000","last_fullsized":"2023-10-14T01:18:05.582364+0000","mapping_epoch":13,"log_start":"128'287916","ondisk_log_start":"128'287916","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":389,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 144s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3590066612,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":287205,"num_read_kb":1114967,"num_write":280381,"num_write_kb":1121167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":338,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:18.595036+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:17:18.595036+0000","last_peered":"2023-10-14T01:17:18.595036+0000","last_clean":"2023-10-14T01:17:18.595036+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:17:18.595036+0000","last_undegraded":"2023-10-14T01:17:18.595036+0000","last_fullsized":"2023-10-14T01:17:18.595036+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28095405100,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2232739,"num_read_kb":8700726,"num_write":2195163,"num_write_kb":8778569,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":17978429440,"data_stored":27854514264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80357,"ondisk_log_size":80357,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739306,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4375044,"kb_used_data":3428156,"kb_used_omap":484,"kb_used_meta":946395,"kb_avail":89996796,"statfs":{"total":96636764160,"available":92156719104,"internally_reserved":0,"allocated":3510431744,"data_stored":5438779852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":969108804},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.79300000000000004,"15min":0.83399999999999996},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.17299999999999999},"max":{"1min":3.964,"5min":6.1120000000000001,"15min":16.474},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.488,"5min":0.99099999999999999,"15min":0.92700000000000005},"min":{"1min":0.219,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":10.871,"5min":10.871,"15min":11.554},"last":0.254}]},{"osd":1,"last update":"Sat Oct 14 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":1.9039999999999999,"5min":2.5409999999999999,"15min":1.821},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.20200000000000001},"max":{"1min":8.8390000000000004,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.32000000000000001},{"interface":"front","average":{"1min":2.012,"5min":2.601,"15min":1.9199999999999999},"min":{"1min":0.39100000000000001,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":8.8819999999999997,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.39600000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739298,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10190820,"kb_used_data":8779796,"kb_used_omap":1127,"kb_used_meta":1409880,"kb_avail":84181020,"statfs":{"total":96636764160,"available":86201364480,"internally_reserved":0,"allocated":8990511104,"data_stored":13928430431,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":1443717558},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,4,6,9,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.84499999999999997,"15min":0.87},"min":{"1min":0.191,"5min":0.191,"15min":0.184},"max":{"1min":4.1180000000000003,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.92400000000000004,"15min":0.74299999999999999},"min":{"1min":0.26200000000000001,"5min":0.252,"15min":0.222},"max":{"1min":3.004,"5min":15.173,"15min":15.173},"last":0.29599999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:17:05 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.995,"15min":0.83799999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.20999999999999999},"max":{"1min":5.9800000000000004,"5min":7.1280000000000001,"15min":10.403},"last":0.33200000000000002},{"interface":"front","average":{"1min":1.206,"5min":0.76500000000000001,"15min":0.877},"min":{"1min":0.22900000000000001,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":7.6299999999999999,"5min":7.6299999999999999,"15min":8.9179999999999993},"last":0.245}]}]},{"osd":0,"up_from":8,"seq":34359739307,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7358992,"kb_used_data":5351176,"kb_used_omap":645,"kb_used_meta":2007162,"kb_avail":87012848,"statfs":{"total":96636764160,"available":89101156352,"internally_reserved":0,"allocated":5479604224,"data_stored":8488519334,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":2055334162},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":1.6759999999999999,"5min":0.98699999999999999,"15min":0.70299999999999996},"min":{"1min":0.20000000000000001,"5min":0.183,"15min":0.182},"max":{"1min":19.274000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":1.7849999999999999,"5min":1.0980000000000001,"15min":0.86199999999999999},"min":{"1min":0.28699999999999998,"5min":0.217,"15min":0.191},"max":{"1min":19.376000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.42699999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:17:04 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":1.141,"15min":1.103},"min":{"1min":0.216,"5min":0.215,"15min":0.185},"max":{"1min":9.3309999999999995,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.91500000000000004,"15min":0.82699999999999996},"min":{"1min":0.23200000000000001,"5min":0.23000000000000001,"15min":0.20999999999999999},"max":{"1min":4.2240000000000002,"5min":7.1440000000000001,"15min":9.5310000000000006},"last":0.39400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5479411712,"data_stored":8488449907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8989573120,"data_stored":13927615532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3509444608,"data_stored":5438448825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:18:10.554 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:10.555 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:10.555 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:10.556 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:10.556 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:10.557 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:18:10.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.618+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:10.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:10.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:10.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:10.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:10.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.642+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:10.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.642+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:10.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:10.642+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:11.366 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:18:11.444 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:18:12.451 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:18:12.508 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:18:13.231 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:18:13.272 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:18:13.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.542+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:13.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:13.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:13.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:13.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:13.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:13.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.558+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:13.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.558+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:13.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.578+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:13.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.666+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:13.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.674+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:13.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:13.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:13.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.690+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:13.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.690+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:13.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:13.690+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:14.245 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:18:14.306 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:18:14.502 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:14.498+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:18:14.528 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:14.526+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:15.008 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:15.006+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:18:15.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.034+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:15.045 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:15.042+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:15.045 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:15.042+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:15.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:15.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:15.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:15.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:15.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.114+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:15.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.114+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:15.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.114+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:15.205 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:18:15.253 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:18:15.339 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:15.334+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:15.360 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:15.358+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:15.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.538+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:15.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.658+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:15.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:15.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:15.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.662+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:15.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.662+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:15.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:15.666+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:18.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.382+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:18.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:18.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:18.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.438+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:18.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.438+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:18.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:18.442+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:21.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:21.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:21.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:21.394+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:21.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:21.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:21.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:21.930+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:23.225 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:23.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:23.251 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:23.250+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:28.443 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:28.442+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:28.454 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:28.450+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:30.159 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:30.154+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:18:30.166 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:30.162+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:30.170 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:30.166+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:30.568 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:30.566+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:30.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.698+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:30.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:30.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:30.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:30.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.798+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:30.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.798+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:30.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.802+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:30.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.942+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:30.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:30.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:30.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.962+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:30.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:30.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.966+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:30.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:30.966+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:32.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:32.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:32.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:32.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:32.420 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:32.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.417+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:32.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:32.719 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:32.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:32.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:32.741+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:33.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.457+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:33.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:33.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:33.473 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:33.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:33.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:33.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.493+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:33.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.493+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:33.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:33.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:33.851 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:33.845+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:34.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:34.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:34.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:34.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:34.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.169+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:34.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.169+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:34.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.169+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:34.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:34.169+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:35.254 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:18:35.940 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:18:35.940 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:18:35.983 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2498,"stamp":"2023-10-14T01:18:34.770248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28137131596,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2242847,"num_read_kb":8740659,"num_write":2205284,"num_write_kb":8820925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80605,"ondisk_log_size":80605,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21699788,"kb_used_data":17638196,"kb_used_omap":2257,"kb_used_meta":4059310,"kb_avail":261415732,"statfs":{"total":289910292480,"available":267689709568,"internally_reserved":0,"allocated":18061512704,"data_stored":27936695249,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2312180,"internal_metadata":4156733452},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,7,9,22,13,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16936960,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4039,"num_read_kb":16120,"num_write":4069,"num_write_kb":16276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.006891"},"pg_stats":[{"pgid":"2.7","version":"130'258704","reported_seq":515246,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:31.513205+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:18:31.513205+0000","last_peered":"2023-10-14T01:18:31.513205+0000","last_clean":"2023-10-14T01:18:31.513205+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:18:31.513205+0000","last_undegraded":"2023-10-14T01:18:31.513205+0000","last_fullsized":"2023-10-14T01:18:31.513205+0000","mapping_epoch":13,"log_start":"127'248611","ondisk_log_start":"127'248611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3275976887,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":257392,"num_read_kb":1014572,"num_write":257648,"num_write_kb":1030577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"130'271757","reported_seq":537189,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:31.505161+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:18:31.505161+0000","last_peered":"2023-10-14T01:18:31.505161+0000","last_clean":"2023-10-14T01:18:31.505161+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:18:31.505161+0000","last_undegraded":"2023-10-14T01:18:31.505161+0000","last_fullsized":"2023-10-14T01:18:31.505161+0000","mapping_epoch":13,"log_start":"128'261708","ondisk_log_start":"128'261708","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3301495051,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":267687,"num_read_kb":1033177,"num_write":258345,"num_write_kb":1032979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"130'277834","reported_seq":552083,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:31.523355+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:18:31.523355+0000","last_peered":"2023-10-14T01:18:31.523355+0000","last_clean":"2023-10-14T01:18:31.523355+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:18:31.523355+0000","last_undegraded":"2023-10-14T01:18:31.523355+0000","last_fullsized":"2023-10-14T01:18:31.523355+0000","mapping_epoch":13,"log_start":"128'267812","ondisk_log_start":"128'267812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":410,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 151s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3471741483,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":275107,"num_read_kb":1080678,"num_write":272275,"num_write_kb":1088979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"130'293825","reported_seq":578158,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:31.506325+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:18:31.506325+0000","last_peered":"2023-10-14T01:18:31.506325+0000","last_clean":"2023-10-14T01:18:31.506325+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:18:31.506325+0000","last_undegraded":"2023-10-14T01:18:31.506325+0000","last_fullsized":"2023-10-14T01:18:31.506325+0000","mapping_epoch":13,"log_start":"128'283811","ondisk_log_start":"128'283811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":704,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 224s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3566731695,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":285321,"num_read_kb":1108075,"num_write":279954,"num_write_kb":1119498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"130'283106","reported_seq":566279,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:30.535811+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:18:30.535811+0000","last_peered":"2023-10-14T01:18:30.535811+0000","last_clean":"2023-10-14T01:18:30.535811+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:18:30.535811+0000","last_undegraded":"2023-10-14T01:18:30.535811+0000","last_fullsized":"2023-10-14T01:18:30.535811+0000","mapping_epoch":13,"log_start":"128'273012","ondisk_log_start":"128'273012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3514130903,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":283579,"num_read_kb":1094007,"num_write":275934,"num_write_kb":1103229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"130'297402","reported_seq":591694,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:30.092452+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:18:30.092452+0000","last_peered":"2023-10-14T01:18:30.092452+0000","last_clean":"2023-10-14T01:18:30.092452+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:18:30.092452+0000","last_undegraded":"2023-10-14T01:18:30.092452+0000","last_fullsized":"2023-10-14T01:18:30.092452+0000","mapping_epoch":13,"log_start":"128'287312","ondisk_log_start":"128'287312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3730768292,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":294871,"num_read_kb":1153812,"num_write":291172,"num_write_kb":1164514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"130'294278","reported_seq":583910,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:18:30.081263+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:18:30.081263+0000","last_peered":"2023-10-14T01:18:30.081263+0000","last_clean":"2023-10-14T01:18:30.081263+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:18:30.081263+0000","last_undegraded":"2023-10-14T01:18:30.081263+0000","last_fullsized":"2023-10-14T01:18:30.081263+0000","mapping_epoch":13,"log_start":"128'284215","ondisk_log_start":"128'284215","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3679916385,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":290273,"num_read_kb":1136126,"num_write":288045,"num_write_kb":1151990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"130'299469","reported_seq":587070,"reported_epoch":130,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:31.494166+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:18:31.494166+0000","last_peered":"2023-10-14T01:18:31.494166+0000","last_clean":"2023-10-14T01:18:31.494166+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:18:31.494166+0000","last_undegraded":"2023-10-14T01:18:31.494166+0000","last_fullsized":"2023-10-14T01:18:31.494166+0000","mapping_epoch":13,"log_start":"128'289416","ondisk_log_start":"128'289416","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":441,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 170s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3595780532,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":288491,"num_read_kb":1120103,"num_write":281742,"num_write_kb":1126611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":338,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:18.595036+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:17:18.595036+0000","last_peered":"2023-10-14T01:17:18.595036+0000","last_clean":"2023-10-14T01:17:18.595036+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:17:18.595036+0000","last_undegraded":"2023-10-14T01:17:18.595036+0000","last_fullsized":"2023-10-14T01:17:18.595036+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28136541228,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2242721,"num_read_kb":8740550,"num_write":2205115,"num_write_kb":8818377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18059771904,"data_stored":27935856728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80478,"ondisk_log_size":80478,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739311,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4468912,"kb_used_data":3443752,"kb_used_omap":484,"kb_used_meta":1024667,"kb_avail":89902928,"statfs":{"total":96636764160,"available":92060598272,"internally_reserved":0,"allocated":3526402048,"data_stored":5454750156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1049259332},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2,5,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.79300000000000004,"15min":0.83399999999999996},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.17299999999999999},"max":{"1min":3.964,"5min":6.1120000000000001,"15min":16.474},"last":0.216},{"interface":"front","average":{"1min":1.488,"5min":0.99099999999999999,"15min":0.92700000000000005},"min":{"1min":0.219,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":10.871,"5min":10.871,"15min":11.554},"last":0.40799999999999997}]},{"osd":1,"last update":"Sat Oct 14 01:17:30 2023","interfaces":[{"interface":"back","average":{"1min":1.9039999999999999,"5min":2.5409999999999999,"15min":1.821},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.20200000000000001},"max":{"1min":8.8390000000000004,"5min":42.481999999999999,"15min":42.481999999999999},"last":0.253},{"interface":"front","average":{"1min":2.012,"5min":2.601,"15min":1.9199999999999999},"min":{"1min":0.39100000000000001,"5min":0.28599999999999998,"15min":0.27000000000000002},"max":{"1min":8.8819999999999997,"5min":42.517000000000003,"15min":42.517000000000003},"last":0.29299999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739303,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10410644,"kb_used_data":8818956,"kb_used_omap":1127,"kb_used_meta":1590552,"kb_avail":83961196,"statfs":{"total":96636764160,"available":85976264704,"internally_reserved":0,"allocated":9030610944,"data_stored":13968530271,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":1628725686},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,4,5,4,12,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":0.94299999999999995,"15min":0.89300000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":7.2560000000000002,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.97499999999999998,"15min":0.76300000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":7.5270000000000001,"5min":15.173,"15min":15.173},"last":0.32400000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.02,"15min":0.872},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":10.403},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.68899999999999995,"15min":0.82799999999999996},"min":{"1min":0.23499999999999999,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":1.3979999999999999,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":0.27900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359739312,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6820232,"kb_used_data":5375488,"kb_used_omap":645,"kb_used_meta":1444090,"kb_avail":87551608,"statfs":{"total":96636764160,"available":89652846592,"internally_reserved":0,"allocated":5504499712,"data_stored":8513414822,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":1478748434},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.97099999999999997,"15min":0.72699999999999998},"min":{"1min":0.245,"5min":0.183,"15min":0.182},"max":{"1min":3.4329999999999998,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":1.0309999999999999,"15min":0.84399999999999997},"min":{"1min":0.24099999999999999,"5min":0.217,"15min":0.21199999999999999},"max":{"1min":2.5920000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.32500000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.153,"15min":1.0760000000000001},"min":{"1min":0.28199999999999997,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":3.6560000000000001,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.438},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.96799999999999997,"15min":0.83899999999999997},"min":{"1min":0.214,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":7.5979999999999999,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":0.28899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5503434752,"data_stored":8512472947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9030815744,"data_stored":13968858156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3525521408,"data_stored":5454525625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:18:35.984 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:35.985 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:35.985 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:35.986 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:35.986 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:35.987 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:18:36.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.309+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:36.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:36.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.325+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:36.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:36.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:36.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.329+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:36.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.329+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:36.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.329+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:36.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.925+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:36.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:36.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:36.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:36.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:36.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.981+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:36.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.981+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:36.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:36.989+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:37.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:37.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:37.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:37.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:37.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:37.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.301+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:37.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.301+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:37.307 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:37.301+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:38.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:38.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:38.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:38.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:38.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.509+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:38.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.509+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:38.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.529+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:38.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.533+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:18:38.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.533+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:18:38.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.565+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:38.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:38.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:38.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:38.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:38.637+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:39.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.017+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:39.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:39.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:39.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:39.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:39.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:39.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.201+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:39.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.201+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:39.217 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:18:39.213+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:18:39.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:39.213+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:39.250 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:18:39.245+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:18:39.250 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:18:39.245+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T01:18:39.251 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:18:39.245+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aff790 2023-10-14T01:18:39.251 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:18:39.245+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69b73ba0 2023-10-14T01:18:39.682 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:39.677+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:39.683 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:39.677+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:40.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.389+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:40.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:40.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.409+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:40.417 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.413+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:40.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:40.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:40.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.437+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:40.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.437+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:40.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:40.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:43.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:43.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:43.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:43.425+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:43.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:43.425+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:43.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:43.425+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:43.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:43.445+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:44.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.109+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:44.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:44.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:44.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:44.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:44.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:44.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.205+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:44.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.205+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:44.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:44.217+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:45.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:45.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:45.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:45.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:45.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:45.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:45.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:45.117+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:45.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:45.117+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:45.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:45.117+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:46.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.593+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:46.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.597+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:46.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:46.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:46.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:46.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.693+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:46.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.709+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:46.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.709+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:46.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:46.721+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:47.495 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:47.489+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:47.496 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:47.489+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:47.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:47.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:47.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:47.861+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:48.452 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:48.449+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:48.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.765+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:48.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:48.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.781+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:48.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:48.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:48.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:48.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:48.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.833+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:48.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.833+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:48.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.833+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:48.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:48.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.841+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:48.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:48.841+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:49.151 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:18:49.145+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:18:49.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:49.429+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:49.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:49.453+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:49.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:49.973+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:49.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:49.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:49.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:49.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:50.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:50.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:50.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:50.005+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:50.096 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:50.093+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:50.134 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:50.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:18:54.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:54.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:54.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:54.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:54.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:54.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:54.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:54.481+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:54.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:54.481+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:54.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:54.493+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:55.988 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:18:56.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:56.105+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:56.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:56.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:56.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:56.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:56.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:56.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:56.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:56.149+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:56.734 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:18:56.735 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:18:56.836 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2508,"stamp":"2023-10-14T01:18:54.777988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28170505804,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2250855,"num_read_kb":8772587,"num_write":2213360,"num_write_kb":8853229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80437,"ondisk_log_size":80437,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22032648,"kb_used_data":17703280,"kb_used_omap":2257,"kb_used_meta":4327086,"kb_avail":261082872,"statfs":{"total":289910292480,"available":267348860928,"internally_reserved":0,"allocated":18128158720,"data_stored":28003328278,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2312180,"internal_metadata":4430936076},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,3,14,13,8,7,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16814080,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3988,"num_read_kb":15904,"num_write":4063,"num_write_kb":16252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006465"},"pg_stats":[{"pgid":"2.7","version":"131'259619","reported_seq":517099,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:18:52.050554+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:18:52.050554+0000","last_peered":"2023-10-14T01:18:52.050554+0000","last_clean":"2023-10-14T01:18:52.050554+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:18:52.050554+0000","last_undegraded":"2023-10-14T01:18:52.050554+0000","last_fullsized":"2023-10-14T01:18:52.050554+0000","mapping_epoch":13,"log_start":"128'249611","ondisk_log_start":"128'249611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3279741111,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":258328,"num_read_kb":1018308,"num_write":258563,"num_write_kb":1034237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"131'272828","reported_seq":539239,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:18:52.018675+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:18:52.018675+0000","last_peered":"2023-10-14T01:18:52.018675+0000","last_clean":"2023-10-14T01:18:52.018675+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:18:52.018675+0000","last_undegraded":"2023-10-14T01:18:52.018675+0000","last_fullsized":"2023-10-14T01:18:52.018675+0000","mapping_epoch":13,"log_start":"128'262808","ondisk_log_start":"128'262808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3305701643,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":268652,"num_read_kb":1037025,"num_write":259356,"num_write_kb":1037023,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"131'278839","reported_seq":554072,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:52.048000+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:18:52.048000+0000","last_peered":"2023-10-14T01:18:52.048000+0000","last_clean":"2023-10-14T01:18:52.048000+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:18:52.048000+0000","last_undegraded":"2023-10-14T01:18:52.048000+0000","last_fullsized":"2023-10-14T01:18:52.048000+0000","mapping_epoch":13,"log_start":"128'268812","ondisk_log_start":"128'268812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":450,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 172s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3475685931,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":276074,"num_read_kb":1084542,"num_write":273232,"num_write_kb":1092807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"131'294961","reported_seq":580303,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:52.045518+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:18:52.045518+0000","last_peered":"2023-10-14T01:18:52.045518+0000","last_clean":"2023-10-14T01:18:52.045518+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:18:52.045518+0000","last_undegraded":"2023-10-14T01:18:52.045518+0000","last_fullsized":"2023-10-14T01:18:52.045518+0000","mapping_epoch":13,"log_start":"128'284911","ondisk_log_start":"128'284911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":745,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 244s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3571098031,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":286307,"num_read_kb":1112003,"num_write":281006,"num_write_kb":1123706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"131'284126","reported_seq":568372,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:18:50.749023+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:18:50.749023+0000","last_peered":"2023-10-14T01:18:50.749023+0000","last_clean":"2023-10-14T01:18:50.749023+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:18:50.749023+0000","last_undegraded":"2023-10-14T01:18:50.749023+0000","last_fullsized":"2023-10-14T01:18:50.749023+0000","mapping_epoch":13,"log_start":"128'274112","ondisk_log_start":"128'274112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3518243287,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":284630,"num_read_kb":1098203,"num_write":276929,"num_write_kb":1107209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"131'298494","reported_seq":593805,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:18:50.363958+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:18:50.363958+0000","last_peered":"2023-10-14T01:18:50.363958+0000","last_clean":"2023-10-14T01:18:50.363958+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:18:50.363958+0000","last_undegraded":"2023-10-14T01:18:50.363958+0000","last_fullsized":"2023-10-14T01:18:50.363958+0000","mapping_epoch":13,"log_start":"128'288412","ondisk_log_start":"128'288412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3735163300,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":295882,"num_read_kb":1157852,"num_write":292246,"num_write_kb":1168810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"131'295341","reported_seq":586028,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:18:50.344873+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:18:50.344873+0000","last_peered":"2023-10-14T01:18:50.344873+0000","last_clean":"2023-10-14T01:18:50.344873+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:18:50.344873+0000","last_undegraded":"2023-10-14T01:18:50.344873+0000","last_fullsized":"2023-10-14T01:18:50.344873+0000","mapping_epoch":13,"log_start":"128'285315","ondisk_log_start":"128'285315","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3684208993,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":291315,"num_read_kb":1140270,"num_write":289090,"num_write_kb":1156170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"131'300599","reported_seq":589278,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:18:52.047271+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:18:52.047271+0000","last_peered":"2023-10-14T01:18:52.047271+0000","last_clean":"2023-10-14T01:18:52.047271+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:18:52.047271+0000","last_undegraded":"2023-10-14T01:18:52.047271+0000","last_fullsized":"2023-10-14T01:18:52.047271+0000","mapping_epoch":13,"log_start":"128'290516","ondisk_log_start":"128'290516","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":480,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 191s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3600073140,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":289541,"num_read_kb":1124275,"num_write":282769,"num_write_kb":1130719,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":339,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:24.496874+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:17:24.496874+0000","last_peered":"2023-10-14T01:17:24.496874+0000","last_clean":"2023-10-14T01:17:24.496874+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:17:24.496874+0000","last_undegraded":"2023-10-14T01:17:24.496874+0000","last_fullsized":"2023-10-14T01:17:24.496874+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28169915436,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2250729,"num_read_kb":8772478,"num_write":2213191,"num_write_kb":8850681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18125705216,"data_stored":28001790040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80310,"ondisk_log_size":80310,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739315,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4519976,"kb_used_data":3456032,"kb_used_omap":484,"kb_used_meta":1063451,"kb_avail":89851864,"statfs":{"total":96636764160,"available":92008308736,"internally_reserved":0,"allocated":3538976768,"data_stored":5467320547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1088974148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.73099999999999998,"15min":0.79400000000000004},"min":{"1min":0.20200000000000001,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":4.1230000000000002,"5min":5.2629999999999999,"15min":7.6390000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.873,"15min":0.94199999999999995},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.17799999999999999},"max":{"1min":3.1859999999999999,"5min":10.871,"15min":11.554},"last":0.504}]},{"osd":1,"last update":"Sat Oct 14 01:18:31 2023","interfaces":[{"interface":"back","average":{"1min":3.3879999999999999,"5min":2.8399999999999999,"15min":1.972},"min":{"1min":0.245,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":44.308,"5min":44.308,"15min":44.308},"last":0.75800000000000001},{"interface":"front","average":{"1min":3.4369999999999998,"5min":2.8999999999999999,"15min":2.0710000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":44.350000000000001,"5min":44.350000000000001,"15min":44.350000000000001},"last":0.79300000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739307,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10596916,"kb_used_data":8852524,"kb_used_omap":1127,"kb_used_meta":1743256,"kb_avail":83774924,"statfs":{"total":96636764160,"available":85785522176,"internally_reserved":0,"allocated":9064984576,"data_stored":14002899574,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":1785094582},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,3,9,6,6,6,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":0.94299999999999995,"15min":0.89300000000000002},"min":{"1min":0.18099999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":7.2560000000000002,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.373},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.97499999999999998,"15min":0.76300000000000001},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":7.5270000000000001,"5min":15.173,"15min":15.173},"last":0.34899999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.02,"15min":0.872},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":10.403},"last":0.317},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.68899999999999995,"15min":0.82799999999999996},"min":{"1min":0.23499999999999999,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":1.3979999999999999,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":3.1440000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739316,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6915756,"kb_used_data":5394724,"kb_used_omap":645,"kb_used_meta":1520378,"kb_avail":87456084,"statfs":{"total":96636764160,"available":89555030016,"internally_reserved":0,"allocated":5524197376,"data_stored":8533108157,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":1556867346},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.97099999999999997,"15min":0.72699999999999998},"min":{"1min":0.245,"5min":0.183,"15min":0.182},"max":{"1min":3.4329999999999998,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":1.0309999999999999,"15min":0.84399999999999997},"min":{"1min":0.24099999999999999,"5min":0.217,"15min":0.21199999999999999},"max":{"1min":2.5920000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.29399999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":1.153,"15min":1.0760000000000001},"min":{"1min":0.28199999999999997,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":3.6560000000000001,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.39400000000000002},{"interface":"front","average":{"1min":1.0449999999999999,"5min":0.96799999999999997,"15min":0.83899999999999997},"min":{"1min":0.214,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":7.5979999999999999,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":0.32900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5523337216,"data_stored":8532375411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9064095744,"data_stored":14002138156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3538272256,"data_stored":5467276473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:18:56.838 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:56.838 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:56.839 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:56.839 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:56.840 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:18:56.840 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:18:57.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.481+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:18:57.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:57.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:57.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:57.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:57.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:57.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.545+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:57.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:57.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:58.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:18:58.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.209+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:18:58.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:18:58.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:18:58.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:18:58.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.257+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:18:58.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.257+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:18:58.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:18:58.265+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:18:58.839 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:58.833+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:18:58.879 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:58.877+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:18:58.894 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:18:58.889+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:00.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.717+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:00.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.725+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:00.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.749+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:00.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:00.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:00.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:00.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.773+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:00.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.773+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:00.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:00.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:01.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:01.449+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:01.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:01.505+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:03.024 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:03.021+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:03.045 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:03.041+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:03.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:03.561+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:03.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:03.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:03.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:03.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:03.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:03.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:03.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:03.609+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:04.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.061+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:04.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:04.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:04.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:04.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:04.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.149+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:04.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.149+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:04.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.149+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:04.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:04.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:04.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:05.150 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:19:05.145+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:19:05.160 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:19:05.157+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:19:05.161 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:19:05.157+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T01:19:05.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:05.617+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:05.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:05.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:05.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:05.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:05.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:05.657+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:06.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:06.369+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:06.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:06.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:06.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:06.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:06.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:06.377+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:07.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:07.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:07.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:07.057+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:09.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:09.253+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:09.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:09.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:09.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:09.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:09.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:09.297+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:12.119 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:12.112+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:12.132 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:12.128+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:12.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.640+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:12.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:12.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:12.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:12.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:12.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.712+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:12.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.712+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:12.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:12.712+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:13.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.288+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:13.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.316+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:13.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:13.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:13.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:13.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.376+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:13.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.376+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:13.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:13.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:13.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.708+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:13.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.728+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:13.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:13.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.736+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:19:13.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.736+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:19:13.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.736+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:13.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:13.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:13.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:13.736+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:16.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:16.748+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:16.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:16.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:16.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:16.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:16.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:16.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:16.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:16.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:16.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:16.804+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:16.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:16.812+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:16.842 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:19:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:19:17.627 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:19:17.676 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2519,"stamp":"2023-10-14T01:19:16.785672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28207140428,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2259611,"num_read_kb":8807503,"num_write":2222268,"num_write_kb":8888861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80527,"ondisk_log_size":80527,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22472052,"kb_used_data":17775196,"kb_used_omap":2257,"kb_used_meta":4694574,"kb_avail":260643468,"statfs":{"total":289910292480,"available":266898911232,"internally_reserved":0,"allocated":18201800704,"data_stored":28076970262,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2312180,"internal_metadata":4807243788},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,4,3,3,9,15,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19910656,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4832,"num_read_kb":19252,"num_write":4843,"num_write_kb":19372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005803"},"pg_stats":[{"pgid":"2.7","version":"131'260545","reported_seq":518948,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:12.568085+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:19:12.568085+0000","last_peered":"2023-10-14T01:19:12.568085+0000","last_clean":"2023-10-14T01:19:12.568085+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:19:12.568085+0000","last_undegraded":"2023-10-14T01:19:12.568085+0000","last_fullsized":"2023-10-14T01:19:12.568085+0000","mapping_epoch":13,"log_start":"128'250511","ondisk_log_start":"128'250511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3283534007,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":259251,"num_read_kb":1021992,"num_write":259489,"num_write_kb":1037941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"131'273864","reported_seq":541255,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:12.577225+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:19:12.577225+0000","last_peered":"2023-10-14T01:19:12.577225+0000","last_clean":"2023-10-14T01:19:12.577225+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:19:12.577225+0000","last_undegraded":"2023-10-14T01:19:12.577225+0000","last_fullsized":"2023-10-14T01:19:12.577225+0000","mapping_epoch":13,"log_start":"128'263808","ondisk_log_start":"128'263808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3309777163,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":269624,"num_read_kb":1040905,"num_write":260352,"num_write_kb":1041007,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"131'279901","reported_seq":556096,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:12.575754+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:19:12.575754+0000","last_peered":"2023-10-14T01:19:12.575754+0000","last_clean":"2023-10-14T01:19:12.575754+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:19:12.575754+0000","last_undegraded":"2023-10-14T01:19:12.575754+0000","last_fullsized":"2023-10-14T01:19:12.575754+0000","mapping_epoch":13,"log_start":"128'269812","ondisk_log_start":"128'269812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":492,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 192s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3479872043,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":277021,"num_read_kb":1088318,"num_write":274253,"num_write_kb":1096891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"131'296128","reported_seq":582465,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:12.586384+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:19:12.586384+0000","last_peered":"2023-10-14T01:19:12.586384+0000","last_clean":"2023-10-14T01:19:12.586384+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:19:12.586384+0000","last_undegraded":"2023-10-14T01:19:12.586384+0000","last_fullsized":"2023-10-14T01:19:12.586384+0000","mapping_epoch":13,"log_start":"128'286111","ondisk_log_start":"128'286111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":786,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 265s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3575620015,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":287282,"num_read_kb":1115883,"num_write":282109,"num_write_kb":1128118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"131'285327","reported_seq":570835,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:16.001962+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:19:16.001962+0000","last_peered":"2023-10-14T01:19:16.001962+0000","last_clean":"2023-10-14T01:19:16.001962+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:19:16.001962+0000","last_undegraded":"2023-10-14T01:19:16.001962+0000","last_fullsized":"2023-10-14T01:19:16.001962+0000","mapping_epoch":13,"log_start":"128'275312","ondisk_log_start":"128'275312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3523178967,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":285869,"num_read_kb":1103155,"num_write":278105,"num_write_kb":1111913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"131'299788","reported_seq":596458,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:15.533038+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:19:15.533038+0000","last_peered":"2023-10-14T01:19:15.533038+0000","last_clean":"2023-10-14T01:19:15.533038+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:19:15.533038+0000","last_undegraded":"2023-10-14T01:19:15.533038+0000","last_fullsized":"2023-10-14T01:19:15.533038+0000","mapping_epoch":13,"log_start":"129'289712","ondisk_log_start":"129'289712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3740397988,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":297230,"num_read_kb":1163232,"num_write":293526,"num_write_kb":1173930,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"131'296745","reported_seq":588744,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:15.524917+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:19:15.524917+0000","last_peered":"2023-10-14T01:19:15.524917+0000","last_clean":"2023-10-14T01:19:15.524917+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:19:15.524917+0000","last_undegraded":"2023-10-14T01:19:15.524917+0000","last_fullsized":"2023-10-14T01:19:15.524917+0000","mapping_epoch":13,"log_start":"128'286715","ondisk_log_start":"128'286715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3689931105,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":292619,"num_read_kb":1145466,"num_write":290480,"num_write_kb":1161730,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"131'301699","reported_seq":591450,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:12.555334+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:19:12.555334+0000","last_peered":"2023-10-14T01:19:12.555334+0000","last_clean":"2023-10-14T01:19:12.555334+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:19:12.555334+0000","last_undegraded":"2023-10-14T01:19:12.555334+0000","last_fullsized":"2023-10-14T01:19:12.555334+0000","mapping_epoch":13,"log_start":"129'291616","ondisk_log_start":"129'291616","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":529,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 211s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3604238772,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":290589,"num_read_kb":1128443,"num_write":283785,"num_write_kb":1134783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":339,"reported_epoch":130,"state":"active+clean","last_fresh":"2023-10-14T01:17:24.496874+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:17:24.496874+0000","last_peered":"2023-10-14T01:17:24.496874+0000","last_clean":"2023-10-14T01:17:24.496874+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:17:24.496874+0000","last_undegraded":"2023-10-14T01:17:24.496874+0000","last_fullsized":"2023-10-14T01:17:24.496874+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28206550060,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2259485,"num_read_kb":8807394,"num_write":2222099,"num_write_kb":8886313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18200477696,"data_stored":28076562520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80400,"ondisk_log_size":80400,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739320,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4625760,"kb_used_data":3471576,"kb_used_omap":484,"kb_used_meta":1153691,"kb_avail":89746080,"statfs":{"total":96636764160,"available":91899985920,"internally_reserved":0,"allocated":3554893824,"data_stored":5483237603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1181379908},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:18:31 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.73099999999999998,"15min":0.79400000000000004},"min":{"1min":0.20200000000000001,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":4.1230000000000002,"5min":5.2629999999999999,"15min":7.6390000000000002},"last":0.627},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.873,"15min":0.94199999999999995},"min":{"1min":0.20200000000000001,"5min":0.20100000000000001,"15min":0.17799999999999999},"max":{"1min":3.1859999999999999,"5min":10.871,"15min":11.554},"last":0.23799999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:18:31 2023","interfaces":[{"interface":"back","average":{"1min":3.3879999999999999,"5min":2.8399999999999999,"15min":1.972},"min":{"1min":0.245,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":44.308,"5min":44.308,"15min":44.308},"last":0.66600000000000004},{"interface":"front","average":{"1min":3.4369999999999998,"5min":2.8999999999999999,"15min":2.0710000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":44.350000000000001,"5min":44.350000000000001,"15min":44.350000000000001},"last":0.69599999999999995}]}]},{"osd":1,"up_from":8,"seq":34359739311,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10758600,"kb_used_data":8884608,"kb_used_omap":1127,"kb_used_meta":1872856,"kb_avail":83613240,"statfs":{"total":96636764160,"available":85619957760,"internally_reserved":0,"allocated":9097838592,"data_stored":14035753590,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":1917804982},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,3,3,1,8,10,2,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.92600000000000005,"15min":0.83699999999999997},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.2130000000000001,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":1.0089999999999999,"15min":0.76400000000000001},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":5.3890000000000002,"5min":15.173,"15min":15.173},"last":0.64500000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:18:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.02,"15min":0.872},"min":{"1min":0.253,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":7.2130000000000001,"5min":7.2130000000000001,"15min":10.403},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.68899999999999995,"15min":0.82799999999999996},"min":{"1min":0.23499999999999999,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":1.3979999999999999,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":0.27900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359739321,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7087692,"kb_used_data":5419012,"kb_used_omap":645,"kb_used_meta":1668026,"kb_avail":87284148,"statfs":{"total":96636764160,"available":89378967552,"internally_reserved":0,"allocated":5549068288,"data_stored":8557979069,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":1708058898},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.1319999999999999,"15min":0.76800000000000002},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.182},"max":{"1min":6.4100000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"last":1.756},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.052,"15min":0.80200000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.21199999999999999},"max":{"1min":2.8500000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.51500000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0609999999999999,"15min":1.0449999999999999},"min":{"1min":0.252,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":6.4530000000000003,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.998,"15min":0.78300000000000003},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":2.3719999999999999,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":0.31900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5548490752,"data_stored":8557528947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9097646080,"data_stored":14035688492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3554340864,"data_stored":5483345081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:19:17.678 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:17.678 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:17.679 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:17.679 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:17.680 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:17.680 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:19:17.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:17.924+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:17.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:17.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:17.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:17.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:17.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:17.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:17.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:17.964+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:17.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:17.964+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:17.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:17.968+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:18.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:18.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:18.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:18.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:18.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:18.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:18.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:18.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:18.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:18.688+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:18.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:18.688+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:18.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:18.688+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:25.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.008+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:25.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:25.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.084+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:25.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.224+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:25.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:25.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.280+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:25.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:25.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.492+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:25.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.532+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:25.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:25.532+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:26.441 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:26.432+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:27.258 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:27.252+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:19:27.264 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:27.260+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:27.264 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:27.260+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:27.265 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:27.260+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:29.388 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:29.384+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:29.395 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:29.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:29.395 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:29.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:30.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.820+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:30.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:30.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.880+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:30.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:30.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:30.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:30.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.916+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:30.977 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.972+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:30.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:30.980+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:31.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:31.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.664+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:31.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:31.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:31.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.680+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:31.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.680+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:31.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.680+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:31.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:31.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.800+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:31.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:31.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:31.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:31.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.864+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:31.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.864+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:31.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.864+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:31.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:31.864+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:32.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:32.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:32.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:32.784+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:33.480 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:33.476+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:33.481 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:33.476+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:33.785 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:33.780+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:33.785 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:33.780+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:33.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:33.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:33.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:33.932+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:36.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:36.448+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:36.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:36.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:37.383 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:37.376+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:37.681 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:19:37.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.900+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:37.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.916+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:37.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.920+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:37.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:37.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:37.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.940+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:37.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.940+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:37.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:37.940+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:38.504 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:19:38.505 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:19:38.583 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2529,"stamp":"2023-10-14T01:19:36.792772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28240473676,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2267673,"num_read_kb":8839635,"num_write":2230340,"num_write_kb":8921149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80594,"ondisk_log_size":80594,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22794416,"kb_used_data":17839444,"kb_used_omap":2257,"kb_used_meta":4952686,"kb_avail":260321104,"statfs":{"total":289910292480,"available":266568810496,"internally_reserved":0,"allocated":18267590656,"data_stored":28142760214,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2312180,"internal_metadata":5071550476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,10,5,39,11,9,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19783680,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4701,"num_read_kb":18736,"num_write":4794,"num_write_kb":19176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005819"},"pg_stats":[{"pgid":"2.7","version":"131'261485","reported_seq":520817,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:33.023819+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:19:33.023819+0000","last_peered":"2023-10-14T01:19:33.023819+0000","last_clean":"2023-10-14T01:19:33.023819+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:19:33.023819+0000","last_undegraded":"2023-10-14T01:19:33.023819+0000","last_fullsized":"2023-10-14T01:19:33.023819+0000","mapping_epoch":13,"log_start":"128'251411","ondisk_log_start":"128'251411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3287417015,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":260181,"num_read_kb":1025684,"num_write":260429,"num_write_kb":1041701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"131'274885","reported_seq":543254,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:33.017086+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:19:33.017086+0000","last_peered":"2023-10-14T01:19:33.017086+0000","last_clean":"2023-10-14T01:19:33.017086+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:19:33.017086+0000","last_undegraded":"2023-10-14T01:19:33.017086+0000","last_fullsized":"2023-10-14T01:19:33.017086+0000","mapping_epoch":13,"log_start":"128'264808","ondisk_log_start":"128'264808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3313828107,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":270585,"num_read_kb":1044737,"num_write":261328,"num_write_kb":1044911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"131'280918","reported_seq":558160,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:32.969651+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:19:32.969651+0000","last_peered":"2023-10-14T01:19:32.969651+0000","last_clean":"2023-10-14T01:19:32.969651+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:19:32.969651+0000","last_undegraded":"2023-10-14T01:19:32.969651+0000","last_fullsized":"2023-10-14T01:19:32.969651+0000","mapping_epoch":13,"log_start":"129'270912","ondisk_log_start":"129'270912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":568,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 213s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3483820587,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":278057,"num_read_kb":1092450,"num_write":275207,"num_write_kb":1100707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"131'297246","reported_seq":584591,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:33.009190+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:19:33.009190+0000","last_peered":"2023-10-14T01:19:33.009190+0000","last_clean":"2023-10-14T01:19:33.009190+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:19:33.009190+0000","last_undegraded":"2023-10-14T01:19:33.009190+0000","last_fullsized":"2023-10-14T01:19:33.009190+0000","mapping_epoch":13,"log_start":"129'287211","ondisk_log_start":"129'287211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":822,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 285s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3579994543,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":288273,"num_read_kb":1119823,"num_write":283159,"num_write_kb":1132318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"131'286390","reported_seq":572890,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:36.141394+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:19:36.141394+0000","last_peered":"2023-10-14T01:19:36.141394+0000","last_clean":"2023-10-14T01:19:36.141394+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:19:36.141394+0000","last_undegraded":"2023-10-14T01:19:36.141394+0000","last_fullsized":"2023-10-14T01:19:36.141394+0000","mapping_epoch":13,"log_start":"129'276312","ondisk_log_start":"129'276312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3527537111,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":286839,"num_read_kb":1107031,"num_write":279153,"num_write_kb":1116105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"131'300840","reported_seq":598498,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:35.750279+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:19:35.750279+0000","last_peered":"2023-10-14T01:19:35.750279+0000","last_clean":"2023-10-14T01:19:35.750279+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:19:35.750279+0000","last_undegraded":"2023-10-14T01:19:35.750279+0000","last_fullsized":"2023-10-14T01:19:35.750279+0000","mapping_epoch":13,"log_start":"129'290812","ondisk_log_start":"129'290812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3744633252,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":298211,"num_read_kb":1167140,"num_write":294566,"num_write_kb":1178090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"131'297796","reported_seq":590947,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:35.690022+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:19:35.690022+0000","last_peered":"2023-10-14T01:19:35.690022+0000","last_clean":"2023-10-14T01:19:35.690022+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:19:35.690022+0000","last_undegraded":"2023-10-14T01:19:35.690022+0000","last_fullsized":"2023-10-14T01:19:35.690022+0000","mapping_epoch":13,"log_start":"129'287715","ondisk_log_start":"129'287715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3694195041,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":293760,"num_read_kb":1150014,"num_write":291519,"num_write_kb":1165886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"131'302804","reported_seq":593617,"reported_epoch":131,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:33.006748+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:19:33.006748+0000","last_peered":"2023-10-14T01:19:33.006748+0000","last_clean":"2023-10-14T01:19:33.006748+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:19:33.006748+0000","last_undegraded":"2023-10-14T01:19:33.006748+0000","last_fullsized":"2023-10-14T01:19:33.006748+0000","mapping_epoch":13,"log_start":"129'292716","ondisk_log_start":"129'292716","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":570,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 232s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3608457652,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":291641,"num_read_kb":1132647,"num_write":284810,"num_write_kb":1138883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":340,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:18:33.608714+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:18:33.608714+0000","last_peered":"2023-10-14T01:18:33.608714+0000","last_clean":"2023-10-14T01:18:33.608714+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:18:33.608714+0000","last_undegraded":"2023-10-14T01:18:33.608714+0000","last_fullsized":"2023-10-14T01:18:33.608714+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28239883308,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2267547,"num_read_kb":8839526,"num_write":2230171,"num_write_kb":8918601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18266222592,"data_stored":28142307416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80467,"ondisk_log_size":80467,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739324,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4695068,"kb_used_data":3484500,"kb_used_omap":484,"kb_used_meta":1210075,"kb_avail":89676772,"statfs":{"total":96636764160,"available":91829014528,"internally_reserved":0,"allocated":3568128000,"data_stored":5496471779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1239117124},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.73199999999999998,"15min":0.755},"min":{"1min":0.20699999999999999,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":4.367,"5min":5.2629999999999999,"15min":6.1120000000000001},"last":0.28899999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.97099999999999997,"15min":0.90200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":7.9160000000000004,"5min":10.871,"15min":11.554},"last":2.891}]},{"osd":1,"last update":"Sat Oct 14 01:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":2.1150000000000002,"15min":1.976},"min":{"1min":0.25600000000000001,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":5.6799999999999997,"5min":44.308,"15min":44.308},"last":0.25600000000000001},{"interface":"front","average":{"1min":1.466,"5min":2.173,"15min":2.0750000000000002},"min":{"1min":0.31900000000000001,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":5.7519999999999998,"5min":44.350000000000001,"15min":44.350000000000001},"last":0.31900000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739315,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10932440,"kb_used_data":8916368,"kb_used_omap":1127,"kb_used_meta":2014936,"kb_avail":83439400,"statfs":{"total":96636764160,"available":85441945600,"internally_reserved":0,"allocated":9130360832,"data_stored":14068275830,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":2063294902},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,6,1,38,3,1,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.92600000000000005,"15min":0.83699999999999997},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.2130000000000001,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.76400000000000001,"5min":1.0089999999999999,"15min":0.76400000000000001},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":5.3890000000000002,"5min":15.173,"15min":15.173},"last":0.29099999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.1180000000000001,"15min":0.93100000000000005},"min":{"1min":0.27600000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.6020000000000003,"5min":9.6020000000000003,"15min":10.403},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77200000000000002,"15min":0.85799999999999998},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":5.5570000000000004,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":0.309}]}]},{"osd":0,"up_from":8,"seq":34359739325,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7166908,"kb_used_data":5438576,"kb_used_omap":645,"kb_used_meta":1727674,"kb_avail":87204932,"statfs":{"total":96636764160,"available":89297850368,"internally_reserved":0,"allocated":5569101824,"data_stored":8578012605,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":1769138450},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.1319999999999999,"15min":0.76800000000000002},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.182},"max":{"1min":6.4100000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.052,"15min":0.80200000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.21199999999999999},"max":{"1min":2.8500000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.30399999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0609999999999999,"15min":1.0449999999999999},"min":{"1min":0.252,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":6.4530000000000003,"5min":9.3309999999999995,"15min":12.622999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.998,"15min":0.78300000000000003},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":2.3719999999999999,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5568471040,"data_stored":8577509235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9130704896,"data_stored":14068747308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3567046656,"data_stored":5496050873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:19:38.585 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:38.585 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:38.586 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:38.586 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:38.587 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:38.587 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:19:39.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:39.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:39.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:39.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:39.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:39.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:39.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:39.320+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:39.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:39.936+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:39.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:39.976+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:39.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:39.988+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:40.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:40.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:40.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:40.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:40.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:40.012+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:40.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:40.012+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:40.015 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:40.012+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:41.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:41.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:41.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:41.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:41.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:41.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:41.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:41.348+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:41.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:41.348+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:41.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:41.348+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:42.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:42.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:42.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:42.588+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:44.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:44.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:44.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:44.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:44.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:44.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:44.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:44.076+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:44.518 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:44.516+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:46.487 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:46.484+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:46.541 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:46.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:48.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.179+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:48.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:48.249 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:48.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.247+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:48.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:48.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:48.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.275+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:48.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.275+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:48.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.307+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:48.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:19:48.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:48.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:48.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:48.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:48.735+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:49.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:49.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:50.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.007+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:50.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.775+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:50.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:50.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:50.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.867+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:50.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.867+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:50.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.867+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:50.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:50.867+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:51.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.343+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:51.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:51.399 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:51.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:51.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.399+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:51.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.399+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:51.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.399+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:51.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:51.403+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:53.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.075+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:19:53.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:53.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:53.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:53.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.127+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:53.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.127+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:53.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.135+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:53.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:53.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:53.481 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:53.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:53.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.495+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:53.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:53.499+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:54.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:54.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:54.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:54.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:54.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:54.967+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:56.236 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:56.235+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:56.242 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:56.239+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:56.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:56.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:56.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:56.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:56.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:56.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:56.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:56.791+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:56.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:56.791+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:57.234 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.231+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:57.245 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.243+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:57.246 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.243+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:57.280 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.279+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:57.295 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.295+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:19:57.301 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.299+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:57.956 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.955+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:57.976 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:57.975+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:58.173 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:58.171+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:19:58.174 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:19:58.171+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:19:58.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.287+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:58.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:58.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.303+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:58.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.303+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:58.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.303+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:58.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.303+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:58.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:19:58.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:19:58.588 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:19:58.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:19:58.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:19:58.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.587+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:19:58.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.595+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:19:58.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:19:58.615+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:19:59.519 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:19:59.519 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:19:59.563 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2541,"stamp":"2023-10-14T01:19:58.799554+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28274515532,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2275958,"num_read_kb":8872687,"num_write":2238538,"num_write_kb":8953941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80481,"ondisk_log_size":80481,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23099072,"kb_used_data":17905576,"kb_used_omap":2257,"kb_used_meta":5191214,"kb_avail":260016448,"statfs":{"total":289910292480,"available":266256842752,"internally_reserved":0,"allocated":18335309824,"data_stored":28210466266,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2312180,"internal_metadata":5315803148},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,9,13,21,24,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17289216,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4143,"num_read_kb":16528,"num_write":4159,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005223"},"pg_stats":[{"pgid":"2.7","version":"132'262425","reported_seq":522737,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:19:53.771140+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:19:53.771140+0000","last_peered":"2023-10-14T01:19:53.771140+0000","last_clean":"2023-10-14T01:19:53.771140+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:19:53.771140+0000","last_undegraded":"2023-10-14T01:19:53.771140+0000","last_fullsized":"2023-10-14T01:19:53.771140+0000","mapping_epoch":13,"log_start":"129'252411","ondisk_log_start":"129'252411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3291279543,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":261159,"num_read_kb":1029580,"num_write":261369,"num_write_kb":1045461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"132'275945","reported_seq":545327,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:19:53.719588+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:19:53.719588+0000","last_peered":"2023-10-14T01:19:53.719588+0000","last_clean":"2023-10-14T01:19:53.719588+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:19:53.719588+0000","last_undegraded":"2023-10-14T01:19:53.719588+0000","last_fullsized":"2023-10-14T01:19:53.719588+0000","mapping_epoch":13,"log_start":"129'265908","ondisk_log_start":"129'265908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3318051083,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":271582,"num_read_kb":1048717,"num_write":262340,"num_write_kb":1048959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"132'282011","reported_seq":560301,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:53.749056+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:19:53.749056+0000","last_peered":"2023-10-14T01:19:53.749056+0000","last_clean":"2023-10-14T01:19:53.749056+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:19:53.749056+0000","last_undegraded":"2023-10-14T01:19:53.749056+0000","last_fullsized":"2023-10-14T01:19:53.749056+0000","mapping_epoch":13,"log_start":"129'271912","ondisk_log_start":"129'271912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":611,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 234s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3488170539,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":279091,"num_read_kb":1096574,"num_write":276254,"num_write_kb":1104895,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"132'298388","reported_seq":586811,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:53.762830+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:19:53.762830+0000","last_peered":"2023-10-14T01:19:53.762830+0000","last_clean":"2023-10-14T01:19:53.762830+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:19:53.762830+0000","last_undegraded":"2023-10-14T01:19:53.762830+0000","last_fullsized":"2023-10-14T01:19:53.762830+0000","mapping_epoch":13,"log_start":"129'288311","ondisk_log_start":"129'288311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":859,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 306s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3584393647,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":289331,"num_read_kb":1124051,"num_write":284230,"num_write_kb":1136602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"132'287374","reported_seq":574906,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:19:56.324698+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:19:56.324698+0000","last_peered":"2023-10-14T01:19:56.324698+0000","last_clean":"2023-10-14T01:19:56.324698+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:19:56.324698+0000","last_undegraded":"2023-10-14T01:19:56.324698+0000","last_fullsized":"2023-10-14T01:19:56.324698+0000","mapping_epoch":13,"log_start":"129'277312","ondisk_log_start":"129'277312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3531571671,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":287853,"num_read_kb":1111071,"num_write":280122,"num_write_kb":1119981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"132'301915","reported_seq":600655,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:19:55.856082+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:19:55.856082+0000","last_peered":"2023-10-14T01:19:55.856082+0000","last_clean":"2023-10-14T01:19:55.856082+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:19:55.856082+0000","last_undegraded":"2023-10-14T01:19:55.856082+0000","last_fullsized":"2023-10-14T01:19:55.856082+0000","mapping_epoch":13,"log_start":"129'291912","ondisk_log_start":"129'291912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3749175716,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":299284,"num_read_kb":1171416,"num_write":295631,"num_write_kb":1182350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"132'298835","reported_seq":593018,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:19:55.848882+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:19:55.848882+0000","last_peered":"2023-10-14T01:19:55.848882+0000","last_clean":"2023-10-14T01:19:55.848882+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:19:55.848882+0000","last_undegraded":"2023-10-14T01:19:55.848882+0000","last_fullsized":"2023-10-14T01:19:55.848882+0000","mapping_epoch":13,"log_start":"129'288815","ondisk_log_start":"129'288815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3698418017,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":294784,"num_read_kb":1154106,"num_write":292548,"num_write_kb":1170002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"132'303958","reported_seq":595914,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:19:53.719493+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:19:53.719493+0000","last_peered":"2023-10-14T01:19:53.719493+0000","last_clean":"2023-10-14T01:19:53.719493+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:19:53.719493+0000","last_undegraded":"2023-10-14T01:19:53.719493+0000","last_fullsized":"2023-10-14T01:19:53.719493+0000","mapping_epoch":13,"log_start":"129'293916","ondisk_log_start":"129'293916","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":609,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 252s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3612864948,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":292748,"num_read_kb":1137063,"num_write":285875,"num_write_kb":1143143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":341,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:27.932935+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:19:27.932935+0000","last_peered":"2023-10-14T01:19:27.932935+0000","last_clean":"2023-10-14T01:19:27.932935+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:19:27.932935+0000","last_undegraded":"2023-10-14T01:19:27.932935+0000","last_fullsized":"2023-10-14T01:19:27.932935+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28273925164,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2275832,"num_read_kb":8872578,"num_write":2238369,"num_write_kb":8951393,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18333437952,"data_stored":28209522776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80354,"ondisk_log_size":80354,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739328,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4739388,"kb_used_data":3496884,"kb_used_omap":484,"kb_used_meta":1242011,"kb_avail":89632452,"statfs":{"total":96636764160,"available":91783630848,"internally_reserved":0,"allocated":3580809216,"data_stored":5509148623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1271819588},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.73199999999999998,"15min":0.755},"min":{"1min":0.20699999999999999,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":4.367,"5min":5.2629999999999999,"15min":6.1120000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.97099999999999997,"15min":0.90200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":7.9160000000000004,"5min":10.871,"15min":11.554},"last":0.56000000000000005}]},{"osd":1,"last update":"Sat Oct 14 01:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":2.1150000000000002,"15min":1.976},"min":{"1min":0.25600000000000001,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":5.6799999999999997,"5min":44.308,"15min":44.308},"last":0.93899999999999995},{"interface":"front","average":{"1min":1.466,"5min":2.173,"15min":2.0750000000000002},"min":{"1min":0.31900000000000001,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":5.7519999999999998,"5min":44.350000000000001,"15min":44.350000000000001},"last":0.96599999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739319,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11096780,"kb_used_data":8950404,"kb_used_omap":1127,"kb_used_meta":2145240,"kb_avail":83275060,"statfs":{"total":96636764160,"available":85273661440,"internally_reserved":0,"allocated":9165213696,"data_stored":14103124322,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1154634,"internal_metadata":2196726198},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,11,8,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:19:09 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.92600000000000005,"15min":0.83699999999999997},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":1.2130000000000001,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.255},{"interface":"front","average":{"1min":0.76400000000000001,"5min":1.0089999999999999,"15min":0.76400000000000001},"min":{"1min":0.26200000000000001,"5min":0.215,"15min":0.215},"max":{"1min":5.3890000000000002,"5min":15.173,"15min":15.173},"last":0.315}]},{"osd":2,"last update":"Sat Oct 14 01:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.1180000000000001,"15min":0.93100000000000005},"min":{"1min":0.27600000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.6020000000000003,"5min":9.6020000000000003,"15min":10.403},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77200000000000002,"15min":0.85799999999999998},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":5.5570000000000004,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":0.33100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739329,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7262904,"kb_used_data":5458288,"kb_used_omap":645,"kb_used_meta":1803962,"kb_avail":87108936,"statfs":{"total":96636764160,"available":89199550464,"internally_reserved":0,"allocated":5589286912,"data_stored":8598193321,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":1847257362},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.1319999999999999,"15min":0.76800000000000002},"min":{"1min":0.20499999999999999,"5min":0.183,"15min":0.182},"max":{"1min":6.4100000000000001,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.60199999999999998,"5min":1.052,"15min":0.80200000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.21199999999999999},"max":{"1min":2.8500000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.88700000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:19:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.0609999999999999,"15min":1.0449999999999999},"min":{"1min":0.252,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":6.4530000000000003,"5min":9.3309999999999995,"15min":12.622999999999999},"last":1.196},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.998,"15min":0.78300000000000003},"min":{"1min":0.24199999999999999,"5min":0.214,"15min":0.20999999999999999},"max":{"1min":2.3719999999999999,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":1.155}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5588844544,"data_stored":8597882739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9164783616,"data_stored":14102826028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3579809792,"data_stored":5508814009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:19:59.564 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:59.565 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:59.565 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:59.566 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:59.566 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:19:59.566 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:20:01.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:01.199+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:01.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:01.239+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:07.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:07.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:07.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:07.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:07.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:07.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:07.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:07.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:07.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:07.251+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:07.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:07.251+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:07.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:07.251+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:08.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:08.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:08.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:08.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:08.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:08.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:08.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:08.435+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:08.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:08.435+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:08.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:08.439+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:09.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.031+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:09.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:09.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:09.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:09.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:09.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:09.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.107+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:09.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.111+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:09.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.111+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:09.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.323+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:09.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:09.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:09.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:09.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:09.419+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:10.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.531+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:10.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:10.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:10.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.559+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:10.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.683+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:10.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:10.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:10.733 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:10.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:10.735 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:10.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:11.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.055+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:11.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:11.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:11.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:11.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.123+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:11.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.843+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:11.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:11.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:11.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.895+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:11.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:11.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.935+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:11.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.935+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:11.946 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:11.943+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:12.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:12.899+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:12.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:12.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:12.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:12.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:12.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:12.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:12.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:12.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:12.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:12.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:13.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.007+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:13.466 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:13.463+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:13.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.687+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:13.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:13.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:13.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:13.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.747+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:13.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.747+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:13.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:13.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.927+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:13.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:13.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:13.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:13.935+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:14.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:20:14.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:20:14.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:14.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:14.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:14.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.439+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:14.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:14.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:14.470 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.463+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:14.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.463+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:14.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.463+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:14.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:14.631+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:15.608 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:15.607+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:16.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:16.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:16.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:16.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:16.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:16.115+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:16.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:16.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:16.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:16.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:16.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:16.119+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:16.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:16.131+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:17.012 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:17.011+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:20:17.022 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:17.019+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:20:17.029 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:17.027+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:17.071 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:17.067+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:19.568 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:20:19.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.635+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:20:19.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.635+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:20:19.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.639+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:19.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:19.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:19.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:19.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.691+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:19.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.691+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:19.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.691+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:19.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:19.691+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:20.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:20.411+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:20.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:20.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:20.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:20.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:20.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:20.431+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:20.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:20.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:20.552 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:20:20.555 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:20:20.615 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2551,"stamp":"2023-10-14T01:20:18.805800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28308016716,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2284105,"num_read_kb":8905183,"num_write":2246603,"num_write_kb":8986201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80673,"ondisk_log_size":80673,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23224528,"kb_used_data":17970360,"kb_used_omap":2333,"kb_used_meta":5251810,"kb_avail":259890992,"statfs":{"total":289910292480,"available":266128375808,"internally_reserved":0,"allocated":18401648640,"data_stored":28276805082,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2389038,"internal_metadata":5377854418},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,12,7,12,19,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16662528,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4143,"num_read_kb":16536,"num_write":4011,"num_write_kb":16044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004947"},"pg_stats":[{"pgid":"2.7","version":"132'263375","reported_seq":524687,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:14.390376+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:20:14.390376+0000","last_peered":"2023-10-14T01:20:14.390376+0000","last_clean":"2023-10-14T01:20:14.390376+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:20:14.390376+0000","last_undegraded":"2023-10-14T01:20:14.390376+0000","last_fullsized":"2023-10-14T01:20:14.390376+0000","mapping_epoch":13,"log_start":"129'253311","ondisk_log_start":"129'253311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3295178935,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":262161,"num_read_kb":1033584,"num_write":262319,"num_write_kb":1049261,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"132'276953","reported_seq":547294,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:14.407395+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:20:14.407395+0000","last_peered":"2023-10-14T01:20:14.407395+0000","last_clean":"2023-10-14T01:20:14.407395+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:20:14.407395+0000","last_undegraded":"2023-10-14T01:20:14.407395+0000","last_fullsized":"2023-10-14T01:20:14.407395+0000","mapping_epoch":13,"log_start":"129'266908","ondisk_log_start":"129'266908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3322048779,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":272530,"num_read_kb":1052497,"num_write":263297,"num_write_kb":1052787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"132'283077","reported_seq":562385,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:20:14.400341+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:20:14.400341+0000","last_peered":"2023-10-14T01:20:14.400341+0000","last_clean":"2023-10-14T01:20:14.400341+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:20:14.400341+0000","last_undegraded":"2023-10-14T01:20:14.400341+0000","last_fullsized":"2023-10-14T01:20:14.400341+0000","mapping_epoch":13,"log_start":"129'273012","ondisk_log_start":"129'273012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":656,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 254s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3492364843,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":280100,"num_read_kb":1100598,"num_write":277274,"num_write_kb":1108975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"132'299480","reported_seq":588967,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:20:14.403835+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:20:14.403835+0000","last_peered":"2023-10-14T01:20:14.403835+0000","last_clean":"2023-10-14T01:20:14.403835+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:20:14.403835+0000","last_undegraded":"2023-10-14T01:20:14.403835+0000","last_fullsized":"2023-10-14T01:20:14.403835+0000","mapping_epoch":13,"log_start":"129'289411","ondisk_log_start":"129'289411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":909,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 327s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3588649391,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":290376,"num_read_kb":1128199,"num_write":285239,"num_write_kb":1140638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"132'288349","reported_seq":576878,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:16.496737+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:20:16.496737+0000","last_peered":"2023-10-14T01:20:16.496737+0000","last_clean":"2023-10-14T01:20:16.496737+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:20:16.496737+0000","last_undegraded":"2023-10-14T01:20:16.496737+0000","last_fullsized":"2023-10-14T01:20:16.496737+0000","mapping_epoch":13,"log_start":"129'278312","ondisk_log_start":"129'278312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3535602135,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":288827,"num_read_kb":1114963,"num_write":281092,"num_write_kb":1123861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"132'303004","reported_seq":602790,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:16.096259+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:20:16.096259+0000","last_peered":"2023-10-14T01:20:16.096259+0000","last_clean":"2023-10-14T01:20:16.096259+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:20:16.096259+0000","last_undegraded":"2023-10-14T01:20:16.096259+0000","last_fullsized":"2023-10-14T01:20:16.096259+0000","mapping_epoch":13,"log_start":"129'292912","ondisk_log_start":"129'292912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3753660836,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":300325,"num_read_kb":1175564,"num_write":296700,"num_write_kb":1186626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"132'299911","reported_seq":595144,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:16.046787+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:20:16.046787+0000","last_peered":"2023-10-14T01:20:16.046787+0000","last_clean":"2023-10-14T01:20:16.046787+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:20:16.046787+0000","last_undegraded":"2023-10-14T01:20:16.046787+0000","last_fullsized":"2023-10-14T01:20:16.046787+0000","mapping_epoch":13,"log_start":"129'289815","ondisk_log_start":"129'289815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3702776161,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":295825,"num_read_kb":1158262,"num_write":293604,"num_write_kb":1174226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"132'305094","reported_seq":598159,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:20:14.339416+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:20:14.339416+0000","last_peered":"2023-10-14T01:20:14.339416+0000","last_clean":"2023-10-14T01:20:14.339416+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:20:14.339416+0000","last_undegraded":"2023-10-14T01:20:14.339416+0000","last_fullsized":"2023-10-14T01:20:14.339416+0000","mapping_epoch":13,"log_start":"129'295016","ondisk_log_start":"129'295016","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":661,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 273s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3617145268,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":293835,"num_read_kb":1141407,"num_write":286909,"num_write_kb":1147279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":341,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:27.932935+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:19:27.932935+0000","last_peered":"2023-10-14T01:19:27.932935+0000","last_clean":"2023-10-14T01:19:27.932935+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:19:27.932935+0000","last_undegraded":"2023-10-14T01:19:27.932935+0000","last_fullsized":"2023-10-14T01:19:27.932935+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28307426348,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2283979,"num_read_kb":8905074,"num_write":2246434,"num_write_kb":8983653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18399358976,"data_stored":28275443800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80546,"ondisk_log_size":80546,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739332,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4818436,"kb_used_data":3509308,"kb_used_omap":484,"kb_used_meta":1308635,"kb_avail":89553404,"statfs":{"total":96636764160,"available":91702685696,"internally_reserved":0,"allocated":3593531392,"data_stored":5521870799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1340042564},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:19:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.73199999999999998,"15min":0.755},"min":{"1min":0.20699999999999999,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":4.367,"5min":5.2629999999999999,"15min":6.1120000000000001},"last":0.26600000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":0.97099999999999997,"15min":0.90200000000000002},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":7.9160000000000004,"5min":10.871,"15min":11.554},"last":0.41399999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:19:35 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":2.1150000000000002,"15min":1.976},"min":{"1min":0.25600000000000001,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":5.6799999999999997,"5min":44.308,"15min":44.308},"last":7.1180000000000003},{"interface":"front","average":{"1min":1.466,"5min":2.173,"15min":2.0750000000000002},"min":{"1min":0.31900000000000001,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":5.7519999999999998,"5min":44.350000000000001,"15min":44.350000000000001},"last":7.1559999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739323,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11030028,"kb_used_data":8983492,"kb_used_omap":1202,"kb_used_meta":2045325,"kb_avail":83341812,"statfs":{"total":96636764160,"available":85342015488,"internally_reserved":0,"allocated":9199095808,"data_stored":14137006434,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1231492,"internal_metadata":2094413180},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,6,8,12,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.0089999999999999,"15min":0.83899999999999997},"min":{"1min":0.20100000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":6.4020000000000001,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.254,"15min":0.84399999999999997},"min":{"1min":0.249,"5min":0.215,"15min":0.215},"max":{"1min":8.4670000000000005,"5min":15.173,"15min":15.173},"last":0.26200000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:19:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4970000000000001,"5min":1.1180000000000001,"15min":0.93100000000000005},"min":{"1min":0.27600000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.6020000000000003,"5min":9.6020000000000003,"15min":10.403},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.77200000000000002,"15min":0.85799999999999998},"min":{"1min":0.25,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":5.5570000000000004,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":0.34200000000000003}]}]},{"osd":0,"up_from":8,"seq":34359739333,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7376064,"kb_used_data":5477560,"kb_used_omap":645,"kb_used_meta":1897850,"kb_avail":86995776,"statfs":{"total":96636764160,"available":89083674624,"internally_reserved":0,"allocated":5609021440,"data_stored":8617927849,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":1943398674},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.111,"15min":0.78700000000000003},"min":{"1min":0.21299999999999999,"5min":0.183,"15min":0.182},"max":{"1min":4.5990000000000002,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.96299999999999997,"15min":0.76400000000000001},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.21199999999999999},"max":{"1min":0.88700000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.29299999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.81499999999999995,"15min":0.95799999999999996},"min":{"1min":0.248,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":1.196,"5min":9.3309999999999995,"15min":9.3309999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91000000000000003,"15min":0.77300000000000002},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":4.6609999999999996,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":0.254}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5608570880,"data_stored":8617609075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9198145536,"data_stored":14136187948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3592642560,"data_stored":5521646777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:20:20.617 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:20.617 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:20.618 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:20.618 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:20.619 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:20.619 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:20:21.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:21.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:21.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:21.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:21.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:21.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:21.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:21.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:21.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:21.187+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:21.555 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:20:21.639 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:20:22.366 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:20:22.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:20:23.279 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:20:23.362 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:20:23.888 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:23.887+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:23.910 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:23.907+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:23.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:23.955+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:24.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:24.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.055+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:24.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:24.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:24.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.099+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:24.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.099+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:24.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.099+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:24.205 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:20:24.293 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:20:24.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.395+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:24.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:24.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:24.452 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:24.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:24.459+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:25.239 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:20:25.331 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:20:26.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.210+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:26.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:26.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.246+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:26.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:26.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:26.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:26.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.270+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:26.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.270+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:26.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:26.270+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:27.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.306+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:27.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:27.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:27.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:27.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:27.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.330+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:27.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.330+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:27.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.334+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:27.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:27.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.514+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:27.525 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.514+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:27.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:27.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:27.710+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:28.784 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.782+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:28.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:28.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:28.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:28.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:28.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.822+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:28.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.822+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:28.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.822+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:28.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.822+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:28.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:28.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:28.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:28.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.882+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:28.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.902+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:28.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:28.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:28.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:28.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:28.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.938+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:28.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.938+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:28.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.938+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:28.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:28.938+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:29.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.782+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:29.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:29.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:29.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:29.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:29.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.858+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:29.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.858+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:29.877 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:29.870+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:30.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:30.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:30.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:30.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:30.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.146+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:30.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.146+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:30.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.146+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:30.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.362+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:30.398 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:30.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:30.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:30.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.418+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:30.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.418+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:30.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:30.446+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:32.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.310+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:32.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:32.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:32.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:32.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:32.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.334+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:32.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.334+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:32.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:32.334+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:33.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:33.522+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:33.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:33.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:33.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:33.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:33.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:33.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:33.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:33.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:33.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:33.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:33.865 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:33.858+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:33.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:33.882+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:34.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:34.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:34.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:34.038+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:34.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:34.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:34.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:34.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:34.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:34.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:35.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.730+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:35.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:35.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:35.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:35.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:35.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.790+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:35.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.790+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:35.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.794+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:35.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:35.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.870+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:35.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:35.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:35.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.886+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:35.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.910+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:35.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:35.910+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:37.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:37.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:37.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:37.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:37.329 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:37.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:37.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:37.342+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:37.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:37.942+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:37.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:37.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:38.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:38.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:38.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:38.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.046+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:38.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.046+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:38.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:38.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:38.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:38.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.098+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:38.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:38.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:38.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.106+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:38.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.106+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:38.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:38.110+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:40.117 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:40.114+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:40.132 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:40.126+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:40.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.482+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:40.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:40.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:40.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:40.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:40.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:40.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.606+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:40.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.606+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:40.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.610+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:40.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.614+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:40.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:40.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:40.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:40.642+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:43.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.462+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:43.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.494+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:43.505 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.502+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:43.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:43.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.510+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:43.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.510+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:43.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.510+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:43.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:43.514+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:44.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.170+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:44.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:44.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:44.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:44.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:44.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.210+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:44.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.210+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:44.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.210+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:44.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:20:44.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:44.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:44.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:44.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:44.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:44.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:44.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:44.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:44.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.750+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:44.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:44.770+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:45.333 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:20:46.135 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:20:46.136 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:20:46.219 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2564,"stamp":"2023-10-14T01:20:44.811095+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28347874892,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2293521,"num_read_kb":8942751,"num_write":2256310,"num_write_kb":9025029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80440,"ondisk_log_size":80440,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23637352,"kb_used_data":18047760,"kb_used_omap":2333,"kb_used_meta":5587234,"kb_avail":259478168,"statfs":{"total":289910292480,"available":265705644032,"internally_reserved":0,"allocated":18480906240,"data_stored":28356062682,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2389038,"internal_metadata":5721328594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,5,9,10,9,17,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16297984,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3778,"num_read_kb":15080,"num_write":3977,"num_write_kb":15908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002652"},"pg_stats":[{"pgid":"2.7","version":"132'264533","reported_seq":526923,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:40.167779+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:20:40.167779+0000","last_peered":"2023-10-14T01:20:40.167779+0000","last_clean":"2023-10-14T01:20:40.167779+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:20:40.167779+0000","last_undegraded":"2023-10-14T01:20:40.167779+0000","last_fullsized":"2023-10-14T01:20:40.167779+0000","mapping_epoch":13,"log_start":"129'254511","ondisk_log_start":"129'254511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3299934391,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":263236,"num_read_kb":1037868,"num_write":263477,"num_write_kb":1053893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"132'278217","reported_seq":549672,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:40.170766+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:20:40.170766+0000","last_peered":"2023-10-14T01:20:40.170766+0000","last_clean":"2023-10-14T01:20:40.170766+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:20:40.170766+0000","last_undegraded":"2023-10-14T01:20:40.170766+0000","last_fullsized":"2023-10-14T01:20:40.170766+0000","mapping_epoch":13,"log_start":"129'268208","ondisk_log_start":"129'268208","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3326943499,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":273628,"num_read_kb":1056881,"num_write":264484,"num_write_kb":1057535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"132'284354","reported_seq":564880,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:20:40.171593+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:20:40.171593+0000","last_peered":"2023-10-14T01:20:40.171593+0000","last_clean":"2023-10-14T01:20:40.171593+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:20:40.171593+0000","last_undegraded":"2023-10-14T01:20:40.171593+0000","last_fullsized":"2023-10-14T01:20:40.171593+0000","mapping_epoch":13,"log_start":"130'274312","ondisk_log_start":"130'274312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":732,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 280s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3497321003,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":281296,"num_read_kb":1105378,"num_write":278476,"num_write_kb":1113783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"132'300815","reported_seq":591544,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:20:40.123273+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:20:40.123273+0000","last_peered":"2023-10-14T01:20:40.123273+0000","last_clean":"2023-10-14T01:20:40.123273+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:20:40.123273+0000","last_undegraded":"2023-10-14T01:20:40.123273+0000","last_fullsized":"2023-10-14T01:20:40.123273+0000","mapping_epoch":13,"log_start":"130'290811","ondisk_log_start":"130'290811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":975,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 352s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3593638319,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":291591,"num_read_kb":1133043,"num_write":286462,"num_write_kb":1145530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"132'289542","reported_seq":579303,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:41.729816+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:20:41.729816+0000","last_peered":"2023-10-14T01:20:41.729816+0000","last_clean":"2023-10-14T01:20:41.729816+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:20:41.729816+0000","last_undegraded":"2023-10-14T01:20:41.729816+0000","last_fullsized":"2023-10-14T01:20:41.729816+0000","mapping_epoch":13,"log_start":"129'279512","ondisk_log_start":"129'279512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3540414935,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":290034,"num_read_kb":1119783,"num_write":282270,"num_write_kb":1128573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"132'304291","reported_seq":605314,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:41.278984+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:20:41.278984+0000","last_peered":"2023-10-14T01:20:41.278984+0000","last_clean":"2023-10-14T01:20:41.278984+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:20:41.278984+0000","last_undegraded":"2023-10-14T01:20:41.278984+0000","last_fullsized":"2023-10-14T01:20:41.278984+0000","mapping_epoch":13,"log_start":"130'294212","ondisk_log_start":"130'294212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3758903716,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":301549,"num_read_kb":1180436,"num_write":297971,"num_write_kb":1191710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"132'301174","reported_seq":597670,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:20:41.228834+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:20:41.228834+0000","last_peered":"2023-10-14T01:20:41.228834+0000","last_clean":"2023-10-14T01:20:41.228834+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:20:41.228834+0000","last_undegraded":"2023-10-14T01:20:41.228834+0000","last_fullsized":"2023-10-14T01:20:41.228834+0000","mapping_epoch":13,"log_start":"130'291115","ondisk_log_start":"130'291115","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3707900257,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":297078,"num_read_kb":1163262,"num_write":294851,"num_write_kb":1179214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"132'306484","reported_seq":600728,"reported_epoch":132,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:20:40.097125+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:20:40.097125+0000","last_peered":"2023-10-14T01:20:40.097125+0000","last_clean":"2023-10-14T01:20:40.097125+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:20:40.097125+0000","last_undegraded":"2023-10-14T01:20:40.097125+0000","last_fullsized":"2023-10-14T01:20:40.097125+0000","mapping_epoch":13,"log_start":"130'296416","ondisk_log_start":"130'296416","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":728,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 299s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3622228404,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":294983,"num_read_kb":1145991,"num_write":288150,"num_write_kb":1152243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":341,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-14T01:19:27.932935+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:19:27.932935+0000","last_peered":"2023-10-14T01:19:27.932935+0000","last_clean":"2023-10-14T01:19:27.932935+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:19:27.932935+0000","last_undegraded":"2023-10-14T01:19:27.932935+0000","last_fullsized":"2023-10-14T01:19:27.932935+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28347284524,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2293395,"num_read_kb":8942642,"num_write":2256141,"num_write_kb":9022481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18478759936,"data_stored":28354844760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80313,"ondisk_log_size":80313,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739337,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4887376,"kb_used_data":3524168,"kb_used_omap":484,"kb_used_meta":1362715,"kb_avail":89484464,"statfs":{"total":96636764160,"available":91632091136,"internally_reserved":0,"allocated":3608748032,"data_stored":5537087439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1395420484},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,0,5,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.66300000000000003,"15min":0.74299999999999999},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":0.57899999999999996,"5min":5.2629999999999999,"15min":6.1120000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.127,"15min":0.97399999999999998},"min":{"1min":0.20699999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":9.3390000000000004,"5min":10.871,"15min":11.554},"last":7.2549999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:20:38 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":2.0030000000000001,"15min":1.9379999999999999},"min":{"1min":0.27400000000000002,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":7.1180000000000003,"5min":44.308,"15min":44.308},"last":2.512},{"interface":"front","average":{"1min":1.673,"5min":2.0609999999999999,"15min":2.0369999999999999},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":7.1559999999999997,"5min":44.350000000000001,"15min":44.350000000000001},"last":2.544}]}]},{"osd":1,"up_from":8,"seq":34359739328,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11264204,"kb_used_data":9022084,"kb_used_omap":1202,"kb_used_meta":2240909,"kb_avail":83107636,"statfs":{"total":96636764160,"available":85102219264,"internally_reserved":0,"allocated":9238614016,"data_stored":14176524642,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1231492,"internal_metadata":2294691196},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,7,4,4,11,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.0089999999999999,"15min":0.83899999999999997},"min":{"1min":0.20100000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":6.4020000000000001,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.25700000000000001},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.254,"15min":0.84399999999999997},"min":{"1min":0.249,"5min":0.215,"15min":0.215},"max":{"1min":8.4670000000000005,"5min":15.173,"15min":15.173},"last":0.38}]},{"osd":2,"last update":"Sat Oct 14 01:20:16 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1850000000000001,"15min":0.96799999999999997},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.0399999999999991,"5min":9.6020000000000003,"15min":10.403},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.83399999999999996},"min":{"1min":0.23100000000000001,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":4.71,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":0.23999999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739338,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7485772,"kb_used_data":5501508,"kb_used_omap":645,"kb_used_meta":1983610,"kb_avail":86886068,"statfs":{"total":96636764160,"available":88971333632,"internally_reserved":0,"allocated":5633544192,"data_stored":8642450601,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":2031216914},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,5,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.111,"15min":0.78700000000000003},"min":{"1min":0.21299999999999999,"5min":0.183,"15min":0.182},"max":{"1min":4.5990000000000002,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.96299999999999997,"15min":0.76400000000000001},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.21199999999999999},"max":{"1min":0.88700000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.50600000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.81499999999999995,"15min":0.95799999999999996},"min":{"1min":0.248,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":1.196,"5min":9.3309999999999995,"15min":9.3309999999999995},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91000000000000003,"15min":0.77300000000000002},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":4.6609999999999996,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":0.46999999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5632417792,"data_stored":8641455987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9238360064,"data_stored":14176402476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3607982080,"data_stored":5536986297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:20:46.221 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:46.221 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:46.222 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:46.222 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:46.223 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:20:46.223 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:20:46.253 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:46.250+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:46.259 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:46.254+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:50.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:50.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:50.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:50.634+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:50.698 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:50.690+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:50.699 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:50.694+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:50.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:50.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:50.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:50.866+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:51.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.310+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:51.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.350+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:51.359 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:51.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.382+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:51.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.382+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:51.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.382+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:51.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.382+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:51.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.390+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:51.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.558+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:51.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.578+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:51.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:51.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:51.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:51.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.642+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:51.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.642+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:51.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:51.642+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:52.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:52.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:52.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:52.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.338+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:52.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.338+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:52.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.338+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:52.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.338+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:52.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:52.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:52.350+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:53.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:53.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:53.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:53.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:53.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:53.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:20:53.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:53.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:20:53.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:53.854+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:20:53.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:53.854+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:20:53.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:53.866+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:54.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:54.426+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:54.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:54.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:54.454 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:54.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:54.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:54.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:55.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:55.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:55.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:55.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.094+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:55.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:55.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.294+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:55.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.306+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:55.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.670+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:55.731 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.718+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:55.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.734+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:55.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:55.742+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:56.033 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:56.022+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:57.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:57.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:57.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:57.714+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:57.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:57.822+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:20:57.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:57.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:20:57.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:57.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:57.858 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:57.854+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:20:57.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:57.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:58.215 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:58.210+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:20:58.227 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:58.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:58.309 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:58.306+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:20:58.341 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:20:58.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:20:59.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:59.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:20:59.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:59.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:20:59.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:20:59.362+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:01.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:01.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:01.246 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:01.238+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:01.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:01.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:01.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:01.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:01.258+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:01.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:01.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:02.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:02.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:02.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:02.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:02.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:02.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:02.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:02.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:02.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:02.106+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:02.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:02.106+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:02.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:02.118+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:03.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:03.306+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:03.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:03.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:03.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:03.330+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:03.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:03.330+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:03.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:03.330+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:03.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:03.330+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:05.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:05.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:05.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:05.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:05.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:05.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:05.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:05.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:05.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:05.170+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:06.225 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:21:06.237 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:06.233+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:21:06.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.233+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:06.247 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:06.245+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:06.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:06.326 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:06.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:06.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:06.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.329+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:06.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.329+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:06.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:06.329+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:07.096 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:21:07.097 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:21:07.183 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2576,"stamp":"2023-10-14T01:21:06.819777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28387499596,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2302961,"num_read_kb":8980431,"num_write":2265983,"num_write_kb":9063721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80584,"ondisk_log_size":80584,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22554352,"kb_used_data":18121484,"kb_used_omap":2334,"kb_used_meta":4430497,"kb_avail":260561168,"statfs":{"total":289910292480,"available":266814636032,"internally_reserved":0,"allocated":18556399616,"data_stored":28431542813,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2390717,"internal_metadata":4536829251},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,11,13,13,12],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16453632,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3786,"num_read_kb":15120,"num_write":4022,"num_write_kb":16088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004940"},"pg_stats":[{"pgid":"2.7","version":"133'265673","reported_seq":529192,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:05.893162+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:21:05.893162+0000","last_peered":"2023-10-14T01:21:05.893162+0000","last_clean":"2023-10-14T01:21:05.893162+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:21:05.893162+0000","last_undegraded":"2023-10-14T01:21:05.893162+0000","last_fullsized":"2023-10-14T01:21:05.893162+0000","mapping_epoch":13,"log_start":"130'255611","ondisk_log_start":"130'255611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3304632503,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":264366,"num_read_kb":1042372,"num_write":264617,"num_write_kb":1058453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"133'279447","reported_seq":552012,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:05.866213+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:21:05.866213+0000","last_peered":"2023-10-14T01:21:05.866213+0000","last_clean":"2023-10-14T01:21:05.866213+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:21:05.866213+0000","last_undegraded":"2023-10-14T01:21:05.866213+0000","last_fullsized":"2023-10-14T01:21:05.866213+0000","mapping_epoch":13,"log_start":"130'269408","ondisk_log_start":"130'269408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3331657995,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":274723,"num_read_kb":1061249,"num_write":265635,"num_write_kb":1062139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"133'285657","reported_seq":567397,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:05.855454+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:21:05.855454+0000","last_peered":"2023-10-14T01:21:05.855454+0000","last_clean":"2023-10-14T01:21:05.855454+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:21:05.855454+0000","last_undegraded":"2023-10-14T01:21:05.855454+0000","last_fullsized":"2023-10-14T01:21:05.855454+0000","mapping_epoch":13,"log_start":"130'275612","ondisk_log_start":"130'275612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":806,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 306s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3502338603,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":282494,"num_read_kb":1110154,"num_write":279705,"num_write_kb":1118699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"133'302185","reported_seq":594222,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:05.867969+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:21:05.867969+0000","last_peered":"2023-10-14T01:21:05.867969+0000","last_clean":"2023-10-14T01:21:05.867969+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:21:05.867969+0000","last_undegraded":"2023-10-14T01:21:05.867969+0000","last_fullsized":"2023-10-14T01:21:05.867969+0000","mapping_epoch":13,"log_start":"130'292112","ondisk_log_start":"130'292112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1056,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 378s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3598737839,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":292873,"num_read_kb":1138159,"num_write":287706,"num_write_kb":1150506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"133'290567","reported_seq":581291,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:01.925260+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:21:01.925260+0000","last_peered":"2023-10-14T01:21:01.925260+0000","last_clean":"2023-10-14T01:21:01.925260+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:21:01.925260+0000","last_undegraded":"2023-10-14T01:21:01.925260+0000","last_fullsized":"2023-10-14T01:21:01.925260+0000","mapping_epoch":13,"log_start":"130'280512","ondisk_log_start":"130'280512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3544572375,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":290978,"num_read_kb":1123551,"num_write":283284,"num_write_kb":1132629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"133'305653","reported_seq":607959,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:06.486538+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:21:06.486538+0000","last_peered":"2023-10-14T01:21:06.486538+0000","last_clean":"2023-10-14T01:21:06.486538+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:21:06.486538+0000","last_undegraded":"2023-10-14T01:21:06.486538+0000","last_fullsized":"2023-10-14T01:21:06.486538+0000","mapping_epoch":13,"log_start":"130'295612","ondisk_log_start":"130'295612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3764400548,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":302821,"num_read_kb":1185516,"num_write":299313,"num_write_kb":1197078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"133'302497","reported_seq":600279,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:06.479683+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:21:06.479683+0000","last_peered":"2023-10-14T01:21:06.479683+0000","last_clean":"2023-10-14T01:21:06.479683+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:21:06.479683+0000","last_undegraded":"2023-10-14T01:21:06.479683+0000","last_fullsized":"2023-10-14T01:21:06.479683+0000","mapping_epoch":13,"log_start":"130'292415","ondisk_log_start":"130'292415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3713237345,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":298353,"num_read_kb":1168358,"num_write":296154,"num_write_kb":1184426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"133'307876","reported_seq":603389,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:05.824764+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:21:05.824764+0000","last_peered":"2023-10-14T01:21:05.824764+0000","last_clean":"2023-10-14T01:21:05.824764+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:21:05.824764+0000","last_undegraded":"2023-10-14T01:21:05.824764+0000","last_fullsized":"2023-10-14T01:21:05.824764+0000","mapping_epoch":13,"log_start":"130'297816","ondisk_log_start":"130'297816","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":795,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 325s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3627332020,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":296227,"num_read_kb":1150963,"num_write":289400,"num_write_kb":1157243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":342,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:19:43.648643+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:19:43.648643+0000","last_peered":"2023-10-14T01:19:43.648643+0000","last_clean":"2023-10-14T01:19:43.648643+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:19:43.648643+0000","last_undegraded":"2023-10-14T01:19:43.648643+0000","last_fullsized":"2023-10-14T01:19:43.648643+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28386909228,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2302835,"num_read_kb":8980322,"num_write":2265814,"num_write_kb":9061173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18554650624,"data_stored":28430735448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80457,"ondisk_log_size":80457,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739342,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4974848,"kb_used_data":3539444,"kb_used_omap":484,"kb_used_meta":1434907,"kb_avail":89396992,"statfs":{"total":96636764160,"available":91542519808,"internally_reserved":0,"allocated":3624390656,"data_stored":5552725648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1469345092},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.66300000000000003,"15min":0.74299999999999999},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":0.57899999999999996,"5min":5.2629999999999999,"15min":6.1120000000000001},"last":0.21199999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.127,"15min":0.97399999999999998},"min":{"1min":0.20699999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":9.3390000000000004,"5min":10.871,"15min":11.554},"last":0.41699999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:20:38 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":2.0030000000000001,"15min":1.9379999999999999},"min":{"1min":0.27400000000000002,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":7.1180000000000003,"5min":44.308,"15min":44.308},"last":0.29099999999999998},{"interface":"front","average":{"1min":1.673,"5min":2.0609999999999999,"15min":2.0369999999999999},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":7.1559999999999997,"5min":44.350000000000001,"15min":44.350000000000001},"last":0.31900000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739333,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9985052,"kb_used_data":9062032,"kb_used_omap":1204,"kb_used_meta":921803,"kb_avail":84386788,"statfs":{"total":96636764160,"available":86412070912,"internally_reserved":0,"allocated":9279520768,"data_stored":14217426979,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":943927021},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,5,11,9,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:20:11 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":1.0089999999999999,"15min":0.83899999999999997},"min":{"1min":0.20100000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":6.4020000000000001,"5min":15.132999999999999,"15min":15.132999999999999},"last":0.27400000000000002},{"interface":"front","average":{"1min":1.7130000000000001,"5min":1.254,"15min":0.84399999999999997},"min":{"1min":0.249,"5min":0.215,"15min":0.215},"max":{"1min":8.4670000000000005,"5min":15.173,"15min":15.173},"last":0.435}]},{"osd":2,"last update":"Sat Oct 14 01:20:16 2023","interfaces":[{"interface":"back","average":{"1min":1.081,"5min":1.1850000000000001,"15min":0.96799999999999997},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":9.0399999999999991,"5min":9.6020000000000003,"15min":10.403},"last":0.315},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.83399999999999996},"min":{"1min":0.23100000000000001,"5min":0.20999999999999999,"15min":0.20100000000000001},"max":{"1min":4.71,"5min":7.6299999999999999,"15min":7.9560000000000004},"last":0.255}]}]},{"osd":0,"up_from":8,"seq":34359739342,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7594452,"kb_used_data":5520008,"kb_used_omap":645,"kb_used_meta":2073786,"kb_avail":86777388,"statfs":{"total":96636764160,"available":88860045312,"internally_reserved":0,"allocated":5652488192,"data_stored":8661390186,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":661230,"internal_metadata":2123557138},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":1.111,"15min":0.78700000000000003},"min":{"1min":0.21299999999999999,"5min":0.183,"15min":0.182},"max":{"1min":4.5990000000000002,"5min":19.274000000000001,"15min":19.274000000000001},"last":3.516},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.96299999999999997,"15min":0.76400000000000001},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.21199999999999999},"max":{"1min":0.88700000000000001,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.25}]},{"osd":2,"last update":"Sat Oct 14 01:20:09 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.81499999999999995,"15min":0.95799999999999996},"min":{"1min":0.248,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":1.196,"5min":9.3309999999999995,"15min":9.3309999999999995},"last":3.5579999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.91000000000000003,"15min":0.77300000000000002},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":4.6609999999999996,"5min":7.5979999999999999,"15min":9.5310000000000006},"last":0.34799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5651693568,"data_stored":8660731763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9279123456,"data_stored":14217165868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3623833600,"data_stored":5552837817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:21:07.184 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:07.185 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:07.185 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:07.185 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:07.186 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:07.186 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:21:07.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.553+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:07.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:07.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.653+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:07.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:07.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:07.668 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.661+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:07.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:07.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.709+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:07.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:07.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:07.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:07.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:08.566 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:08.561+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:11.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.153+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:11.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:11.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:11.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:11.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:11.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:11.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:11.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.217+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:11.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:11.217+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:11.578 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:11.573+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:21:11.596 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:11.593+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:12.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:12.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:12.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.177+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:12.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:12.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.197+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:12.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.197+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:12.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.197+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:12.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.585+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:12.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.641+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:12.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:12.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:12.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:12.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:12.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:12.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:12.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:12.681+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:13.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:13.497+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:13.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:13.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:13.535 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:13.529+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:13.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:13.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:13.557 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:13.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:13.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:13.553+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:13.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:13.553+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:13.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:13.553+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:15.340 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:15.337+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:21:15.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:15.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:15.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:15.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:15.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:15.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:15.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:15.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:15.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:15.381+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:15.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:15.381+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:15.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:15.381+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:15.402 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:15.397+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:17.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:17.941+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:17.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:17.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:18.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:18.021+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:22.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:22.765+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:22.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:22.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:22.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:22.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:22.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:22.805+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:22.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:22.821+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:23.059 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:23.053+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:23.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:23.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:23.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:23.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:23.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:23.377+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:25.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.105+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:25.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:25.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:25.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.189+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:25.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.825+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:25.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.829+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:25.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:25.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:25.873+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:26.262 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:26.257+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:21:26.264 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:26.261+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:27.187 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:21:28.156 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:21:28.157 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:21:28.225 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2586,"stamp":"2023-10-14T01:21:26.826132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28420484684,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2310923,"num_read_kb":9012191,"num_write":2273975,"num_write_kb":9095689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80312,"ondisk_log_size":80312,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":21886024,"kb_used_data":18184748,"kb_used_omap":2407,"kb_used_meta":3698840,"kb_avail":261229496,"statfs":{"total":289910292480,"available":267499003904,"internally_reserved":0,"allocated":18621181952,"data_stored":28496325149,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":3787612816},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,7,9,15,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23908352,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5814,"num_read_kb":23216,"num_write":5781,"num_write_kb":23124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004641"},"pg_stats":[{"pgid":"2.7","version":"133'266581","reported_seq":531043,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:26.279803+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:21:26.279803+0000","last_peered":"2023-10-14T01:21:26.279803+0000","last_clean":"2023-10-14T01:21:26.279803+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:21:26.279803+0000","last_undegraded":"2023-10-14T01:21:26.279803+0000","last_fullsized":"2023-10-14T01:21:26.279803+0000","mapping_epoch":13,"log_start":"130'256511","ondisk_log_start":"130'256511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3308413111,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":265308,"num_read_kb":1046116,"num_write":265525,"num_write_kb":1062085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"133'280431","reported_seq":553922,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:26.305865+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:21:26.305865+0000","last_peered":"2023-10-14T01:21:26.305865+0000","last_clean":"2023-10-14T01:21:26.305865+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:21:26.305865+0000","last_undegraded":"2023-10-14T01:21:26.305865+0000","last_fullsized":"2023-10-14T01:21:26.305865+0000","mapping_epoch":13,"log_start":"130'270408","ondisk_log_start":"130'270408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3335586059,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":275637,"num_read_kb":1064897,"num_write":266580,"num_write_kb":1065919,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"133'286715","reported_seq":569455,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:26.306475+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:21:26.306475+0000","last_peered":"2023-10-14T01:21:26.306475+0000","last_clean":"2023-10-14T01:21:26.306475+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:21:26.306475+0000","last_undegraded":"2023-10-14T01:21:26.306475+0000","last_fullsized":"2023-10-14T01:21:26.306475+0000","mapping_epoch":13,"log_start":"130'276712","ondisk_log_start":"130'276712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":893,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 326s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3506430507,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":283483,"num_read_kb":1114106,"num_write":280692,"num_write_kb":1122647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"133'303321","reported_seq":596411,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:26.289867+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:21:26.289867+0000","last_peered":"2023-10-14T01:21:26.289867+0000","last_clean":"2023-10-14T01:21:26.289867+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:21:26.289867+0000","last_undegraded":"2023-10-14T01:21:26.289867+0000","last_fullsized":"2023-10-14T01:21:26.289867+0000","mapping_epoch":13,"log_start":"130'293312","ondisk_log_start":"130'293312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1125,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 399s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3603050927,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":293908,"num_read_kb":1142291,"num_write":288756,"num_write_kb":1154706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"133'291629","reported_seq":583343,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:22.098659+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:21:22.098659+0000","last_peered":"2023-10-14T01:21:22.098659+0000","last_clean":"2023-10-14T01:21:22.098659+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:21:22.098659+0000","last_undegraded":"2023-10-14T01:21:22.098659+0000","last_fullsized":"2023-10-14T01:21:22.098659+0000","mapping_epoch":13,"log_start":"130'281612","ondisk_log_start":"130'281612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3548840407,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":291947,"num_read_kb":1127415,"num_write":284321,"num_write_kb":1136777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"133'306714","reported_seq":610069,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:26.659749+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:21:26.659749+0000","last_peered":"2023-10-14T01:21:26.659749+0000","last_clean":"2023-10-14T01:21:26.659749+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:21:26.659749+0000","last_undegraded":"2023-10-14T01:21:26.659749+0000","last_fullsized":"2023-10-14T01:21:26.659749+0000","mapping_epoch":13,"log_start":"130'296712","ondisk_log_start":"130'296712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3768717732,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":303861,"num_read_kb":1189660,"num_write":300358,"num_write_kb":1201258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"133'303543","reported_seq":602384,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:26.658806+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:21:26.658806+0000","last_peered":"2023-10-14T01:21:26.658806+0000","last_clean":"2023-10-14T01:21:26.658806+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:21:26.658806+0000","last_undegraded":"2023-10-14T01:21:26.658806+0000","last_fullsized":"2023-10-14T01:21:26.658806+0000","mapping_epoch":13,"log_start":"130'293515","ondisk_log_start":"130'293515","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3717472609,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":299405,"num_read_kb":1172558,"num_write":297184,"num_write_kb":1188546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"133'308949","reported_seq":605511,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:26.244610+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:21:26.244610+0000","last_peered":"2023-10-14T01:21:26.244610+0000","last_clean":"2023-10-14T01:21:26.244610+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:21:26.244610+0000","last_undegraded":"2023-10-14T01:21:26.244610+0000","last_fullsized":"2023-10-14T01:21:26.244610+0000","mapping_epoch":13,"log_start":"130'298916","ondisk_log_start":"130'298916","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":843,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 345s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3631382964,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":297248,"num_read_kb":1155039,"num_write":290390,"num_write_kb":1161203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":342,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-10-14T01:19:43.648643+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:19:43.648643+0000","last_peered":"2023-10-14T01:19:43.648643+0000","last_clean":"2023-10-14T01:19:43.648643+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:19:43.648643+0000","last_undegraded":"2023-10-14T01:19:43.648643+0000","last_fullsized":"2023-10-14T01:19:43.648643+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28419894316,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2310797,"num_read_kb":9012082,"num_write":2273806,"num_write_kb":9093141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18620203008,"data_stored":28496287832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80185,"ondisk_log_size":80185,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739346,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5055080,"kb_used_data":3551708,"kb_used_omap":484,"kb_used_meta":1502875,"kb_avail":89316760,"statfs":{"total":96636764160,"available":91460362240,"internally_reserved":0,"allocated":3636948992,"data_stored":5565283984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1538944324},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:20:38 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.66300000000000003,"15min":0.74299999999999999},"min":{"1min":0.214,"5min":0.19600000000000001,"15min":0.17699999999999999},"max":{"1min":0.57899999999999996,"5min":5.2629999999999999,"15min":6.1120000000000001},"last":3.1869999999999998},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.127,"15min":0.97399999999999998},"min":{"1min":0.20699999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":9.3390000000000004,"5min":10.871,"15min":11.554},"last":7.2249999999999996}]},{"osd":1,"last update":"Sat Oct 14 01:20:38 2023","interfaces":[{"interface":"back","average":{"1min":1.619,"5min":2.0030000000000001,"15min":1.9379999999999999},"min":{"1min":0.27400000000000002,"5min":0.22800000000000001,"15min":0.20699999999999999},"max":{"1min":7.1180000000000003,"5min":44.308,"15min":44.308},"last":0.435},{"interface":"front","average":{"1min":1.673,"5min":2.0609999999999999,"15min":2.0369999999999999},"min":{"1min":0.30099999999999999,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":7.1559999999999997,"5min":44.350000000000001,"15min":44.350000000000001},"last":0.54900000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739337,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10132964,"kb_used_data":9093340,"kb_used_omap":1204,"kb_used_meta":1038411,"kb_avail":84238876,"statfs":{"total":96636764160,"available":86260609024,"internally_reserved":0,"allocated":9311580160,"data_stored":14249486371,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1063333613},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,6,7,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.70099999999999996,"15min":0.79600000000000004},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":3.8690000000000002,"5min":7.2560000000000002,"15min":15.132999999999999},"last":0.23000000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.0609999999999999,"15min":0.88900000000000001},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":5.2690000000000001,"5min":8.4670000000000005,"15min":15.173},"last":0.48299999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":1.0309999999999999,"15min":0.97099999999999997},"min":{"1min":0.27400000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.8109999999999999,"5min":9.6020000000000003,"15min":10.403},"last":8.1809999999999992},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":4.069,"5min":7.6299999999999999,"15min":7.6299999999999999},"last":8.202}]}]},{"osd":0,"up_from":8,"seq":34359739347,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6697980,"kb_used_data":5539700,"kb_used_omap":718,"kb_used_meta":1157553,"kb_avail":87673860,"statfs":{"total":96636764160,"available":89778032640,"internally_reserved":0,"allocated":5672652800,"data_stored":8681554794,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1185334879},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:21:11 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.3180000000000001,"15min":0.85899999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.182},"max":{"1min":7.9459999999999997,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.875,"15min":0.745},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.6549999999999998,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.41799999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.75800000000000001,"15min":0.92400000000000004},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":3.5579999999999998,"5min":9.3309999999999995,"15min":9.3309999999999995},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.75700000000000001,"15min":0.72499999999999998},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":4.2439999999999998,"5min":7.5979999999999999,"15min":7.5979999999999999},"last":0.26100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5671833600,"data_stored":8680871795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9312047104,"data_stored":14250089516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3636322304,"data_stored":5565326521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:21:28.227 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:28.227 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:28.228 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:28.228 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:28.229 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:28.229 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:21:29.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.493+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:29.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:29.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:29.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:29.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:29.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.529+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:29.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.529+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:29.537 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.529+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:29.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.597+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:29.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:29.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:29.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:29.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.629+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:29.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.629+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:29.644 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:29.641+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:30.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.145+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:30.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:30.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:30.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:30.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:30.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.173+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:30.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.173+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:30.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:30.173+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:31.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.649+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:31.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:31.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:31.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.757+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:31.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.973+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:31.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:31.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:31.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:31.993+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:33.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.093+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:33.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:33.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:33.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:33.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:33.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:33.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:33.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:33.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:33.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:33.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:33.669+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:35.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:34.997+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:35.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:35.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:35.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:35.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:35.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:35.033+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:35.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:35.093+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:36.885 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:36.881+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:21:36.886 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:36.881+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:21:36.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:36.885+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:21:36.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:36.885+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:37.679 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:37.669+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:37.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:37.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:37.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:37.701+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:38.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:38.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:38.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:38.865+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:38.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:38.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:38.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:38.885+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:38.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:38.885+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:38.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:38.885+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:38.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:38.917+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:40.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:40.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:40.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:40.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.101+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:40.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.101+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:40.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.109+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:40.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:40.402 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:40.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:40.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:40.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.421+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:40.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.421+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:40.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:40.465+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:41.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:41.257+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:41.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:41.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:41.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:41.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:41.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:41.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:41.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:41.305+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:41.309 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:41.305+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:41.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:41.317+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:41.598 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:41.593+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:21:41.620 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:41.617+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:43.690 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:43.681+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:21:43.704 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:43.697+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:21:46.101 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:46.096+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:21:46.105 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:46.100+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:47.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:47.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:47.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:47.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:47.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:47.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:47.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:47.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:47.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:47.328+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:47.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:47.328+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:47.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:47.328+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:48.230 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:21:48.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:48.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:48.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:48.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:48.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.648+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:48.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.648+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:48.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.660+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:48.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:48.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:49.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.028+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:49.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:49.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:49.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.112+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:49.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:49.192 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:21:49.193 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:21:49.243 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2597,"stamp":"2023-10-14T01:21:48.838528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28453752396,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2318874,"num_read_kb":9043887,"num_write":2282056,"num_write_kb":9128013,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80383,"ondisk_log_size":80383,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22035788,"kb_used_data":18252724,"kb_used_omap":2407,"kb_used_meta":3780632,"kb_avail":261079732,"statfs":{"total":289910292480,"available":267345645568,"internally_reserved":0,"allocated":18690789376,"data_stored":28565932573,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":3871367824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,12,7,8,22,15,7],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17629184,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4170,"num_read_kb":16612,"num_write":4276,"num_write_kb":17104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006410"},"pg_stats":[{"pgid":"2.7","version":"133'267519","reported_seq":532826,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:46.811270+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:21:46.811270+0000","last_peered":"2023-10-14T01:21:46.811270+0000","last_clean":"2023-10-14T01:21:46.811270+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:21:46.811270+0000","last_undegraded":"2023-10-14T01:21:46.811270+0000","last_fullsized":"2023-10-14T01:21:46.811270+0000","mapping_epoch":13,"log_start":"130'257511","ondisk_log_start":"130'257511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3312263351,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":266151,"num_read_kb":1049488,"num_write":266463,"num_write_kb":1065837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"133'281420","reported_seq":555850,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:46.797004+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:21:46.797004+0000","last_peered":"2023-10-14T01:21:46.797004+0000","last_clean":"2023-10-14T01:21:46.797004+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:21:46.797004+0000","last_undegraded":"2023-10-14T01:21:46.797004+0000","last_fullsized":"2023-10-14T01:21:46.797004+0000","mapping_epoch":13,"log_start":"130'271408","ondisk_log_start":"130'271408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3339448587,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":276564,"num_read_kb":1068593,"num_write":267527,"num_write_kb":1069707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"133'287738","reported_seq":571443,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:46.766986+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:21:46.766986+0000","last_peered":"2023-10-14T01:21:46.766986+0000","last_clean":"2023-10-14T01:21:46.766986+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:21:46.766986+0000","last_undegraded":"2023-10-14T01:21:46.766986+0000","last_fullsized":"2023-10-14T01:21:46.766986+0000","mapping_epoch":13,"log_start":"130'277712","ondisk_log_start":"130'277712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":949,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 347s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3510420011,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":284442,"num_read_kb":1117926,"num_write":281665,"num_write_kb":1126539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"133'304379","reported_seq":598466,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:46.745883+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:21:46.745883+0000","last_peered":"2023-10-14T01:21:46.745883+0000","last_clean":"2023-10-14T01:21:46.745883+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:21:46.745883+0000","last_undegraded":"2023-10-14T01:21:46.745883+0000","last_fullsized":"2023-10-14T01:21:46.745883+0000","mapping_epoch":13,"log_start":"131'294312","ondisk_log_start":"131'294312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1152,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 419s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3607187887,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":294890,"num_read_kb":1146199,"num_write":289754,"num_write_kb":1158698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"133'292860","reported_seq":585847,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:47.469920+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:21:47.469920+0000","last_peered":"2023-10-14T01:21:47.469920+0000","last_clean":"2023-10-14T01:21:47.469920+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:21:47.469920+0000","last_undegraded":"2023-10-14T01:21:47.469920+0000","last_fullsized":"2023-10-14T01:21:47.469920+0000","mapping_epoch":13,"log_start":"130'282812","ondisk_log_start":"130'282812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3553845719,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":293193,"num_read_kb":1132387,"num_write":285527,"num_write_kb":1141601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"133'307717","reported_seq":612112,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:46.727251+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:21:46.727251+0000","last_peered":"2023-10-14T01:21:46.727251+0000","last_clean":"2023-10-14T01:21:46.727251+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:21:46.727251+0000","last_undegraded":"2023-10-14T01:21:46.727251+0000","last_fullsized":"2023-10-14T01:21:46.727251+0000","mapping_epoch":13,"log_start":"131'297712","ondisk_log_start":"131'297712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3772760484,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":304900,"num_read_kb":1193804,"num_write":301347,"num_write_kb":1205214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"133'304578","reported_seq":604390,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:46.744175+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:21:46.744175+0000","last_peered":"2023-10-14T01:21:46.744175+0000","last_clean":"2023-10-14T01:21:46.744175+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:21:46.744175+0000","last_undegraded":"2023-10-14T01:21:46.744175+0000","last_fullsized":"2023-10-14T01:21:46.744175+0000","mapping_epoch":13,"log_start":"131'294515","ondisk_log_start":"131'294515","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3721658721,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":300366,"num_read_kb":1176382,"num_write":298205,"num_write_kb":1192630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"133'310043","reported_seq":607623,"reported_epoch":133,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:21:46.746493+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:21:46.746493+0000","last_peered":"2023-10-14T01:21:46.746493+0000","last_clean":"2023-10-14T01:21:46.746493+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:21:46.746493+0000","last_undegraded":"2023-10-14T01:21:46.746493+0000","last_fullsized":"2023-10-14T01:21:46.746493+0000","mapping_epoch":13,"log_start":"131'300016","ondisk_log_start":"131'300016","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":891,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 365s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3635577268,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":298242,"num_read_kb":1158999,"num_write":291399,"num_write_kb":1165239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":343,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:20:58.668709+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:20:58.668709+0000","last_peered":"2023-10-14T01:20:58.668709+0000","last_clean":"2023-10-14T01:20:58.668709+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:20:58.668709+0000","last_undegraded":"2023-10-14T01:20:58.668709+0000","last_fullsized":"2023-10-14T01:20:58.668709+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28453162028,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2318748,"num_read_kb":9043778,"num_write":2281887,"num_write_kb":9125465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18689298432,"data_stored":28565383256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80256,"ondisk_log_size":80256,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739350,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5080844,"kb_used_data":3563588,"kb_used_omap":484,"kb_used_meta":1516763,"kb_avail":89290996,"statfs":{"total":96636764160,"available":91433979904,"internally_reserved":0,"allocated":3649114112,"data_stored":5577449104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1553165636},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1,4,7,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.182,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":3.9180000000000001,"5min":4.367,"15min":6.1120000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.9570000000000001,"5min":1.3220000000000001,"15min":1.0569999999999999},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":11.481999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.434}]},{"osd":1,"last update":"Sat Oct 14 01:21:41 2023","interfaces":[{"interface":"back","average":{"1min":1.8260000000000001,"5min":2.032,"15min":1.984},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":9.9900000000000002,"5min":44.308,"15min":44.308},"last":1.984},{"interface":"front","average":{"1min":1.8899999999999999,"5min":2.0950000000000002,"15min":2.085},"min":{"1min":0.313,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":10.026,"5min":44.350000000000001,"15min":44.350000000000001},"last":2.0139999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739341,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10290956,"kb_used_data":9125828,"kb_used_omap":1204,"kb_used_meta":1163915,"kb_avail":84080884,"statfs":{"total":96636764160,"available":86098825216,"internally_reserved":0,"allocated":9344847872,"data_stored":14282754083,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1191849709},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,8,3,1,12,7,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.70099999999999996,"15min":0.79600000000000004},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":3.8690000000000002,"5min":7.2560000000000002,"15min":15.132999999999999},"last":0.318},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.0609999999999999,"15min":0.88900000000000001},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":5.2690000000000001,"5min":8.4670000000000005,"15min":15.173},"last":0.27800000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":1.0309999999999999,"15min":0.97099999999999997},"min":{"1min":0.27400000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.8109999999999999,"5min":9.6020000000000003,"15min":10.403},"last":8.2409999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":4.069,"5min":7.6299999999999999,"15min":7.6299999999999999},"last":0.25800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739352,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6663988,"kb_used_data":5563308,"kb_used_omap":718,"kb_used_meta":1099953,"kb_avail":87707852,"statfs":{"total":96636764160,"available":89812840448,"internally_reserved":0,"allocated":5696827392,"data_stored":8705729386,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1126352479},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,2,6,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:21:11 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.3180000000000001,"15min":0.85899999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.182},"max":{"1min":7.9459999999999997,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.875,"15min":0.745},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.6549999999999998,"5min":19.376000000000001,"15min":19.376000000000001},"last":5.2169999999999996}]},{"osd":2,"last update":"Sat Oct 14 01:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.75800000000000001,"15min":0.92400000000000004},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":3.5579999999999998,"5min":9.3309999999999995,"15min":9.3309999999999995},"last":0.248},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.75700000000000001,"15min":0.72499999999999998},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":4.2439999999999998,"5min":7.5979999999999999,"15min":7.5979999999999999},"last":5.1719999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5696372736,"data_stored":8705410931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9344319488,"data_stored":14282361900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3648606208,"data_stored":5577610425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:21:49.245 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:49.245 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:49.246 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:49.246 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:49.247 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:21:49.247 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:21:49.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.520+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:49.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.548+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:49.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:49.565 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:49.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:49.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.608+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:49.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.608+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:49.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:49.696+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:51.598 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:51.592+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:21:51.620 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:51.616+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:21:51.620 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:51.616+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:53.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:53.084+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:53.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:53.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:53.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:53.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:53.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:53.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:53.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:53.168+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:54.100 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:54.096+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:21:54.114 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:54.108+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:54.759 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:54.752+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:21:57.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:57.728+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:57.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:57.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:57.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:57.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:57.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:57.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:57.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:57.756+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:57.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:57.756+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:59.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.252+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:59.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:21:59.299 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:21:59.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:59.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:59.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:59.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.332+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:59.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.332+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:59.341 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:59.332+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:21:59.349 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:21:59.344+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:21:59.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.368+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:21:59.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.416+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:21:59.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.444+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:21:59.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:21:59.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:21:59.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.456+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:21:59.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.456+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:21:59.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:21:59.472+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:00.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:00.636+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:00.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:00.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:00.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:00.700+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:00.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:00.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:00.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:00.944+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:01.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.032+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:01.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.056+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:01.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.440+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:01.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:01.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.528+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:01.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.528+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:01.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.528+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:01.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.540+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:01.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.800+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:01.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:01.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:01.842 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:01.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:01.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.860+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:01.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.860+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:01.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.860+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:01.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.860+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:22:01.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.860+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:22:01.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.872+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:01.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:01.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:01.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:01.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:01.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:01.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.908+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:01.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:01.908+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:02.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:02.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:02.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:02.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:02.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:02.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:02.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:02.104+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:03.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:03.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:03.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:03.752+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:03.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:03.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:03.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:03.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:03.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:03.776+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:04.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:04.604+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:04.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:04.612+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:04.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:04.688+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:04.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:04.688+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:04.702 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:04.696+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:06.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:06.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:06.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:06.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:06.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:06.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:06.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:06.800+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:06.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:06.800+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:07.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:07.100+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:07.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:07.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:07.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:07.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:07.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:07.128+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:07.358 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:07.352+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:07.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:07.356+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:08.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:08.476+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:08.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:08.508+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:08.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:08.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:08.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:08.520+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:08.535 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:08.524+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:09.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:09.104+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:09.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:09.124+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:09.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:22:10.181 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:22:10.182 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:22:10.214 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2608,"stamp":"2023-10-14T01:22:08.846878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28486606412,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2326793,"num_read_kb":9075483,"num_write":2290073,"num_write_kb":9160081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80426,"ondisk_log_size":80426,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22374392,"kb_used_data":18317408,"kb_used_omap":2407,"kb_used_meta":4054552,"kb_avail":260741128,"statfs":{"total":289910292480,"available":266998915072,"internally_reserved":0,"allocated":18757025792,"data_stored":28632155615,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":4151861904},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,7,8,14,19,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16248832,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3807,"num_read_kb":15180,"num_write":3966,"num_write_kb":15864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.164861"},"pg_stats":[{"pgid":"2.7","version":"134'268470","reported_seq":534724,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:07.252006+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:22:07.252006+0000","last_peered":"2023-10-14T01:22:07.252006+0000","last_clean":"2023-10-14T01:22:07.252006+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:22:07.252006+0000","last_undegraded":"2023-10-14T01:22:07.252006+0000","last_fullsized":"2023-10-14T01:22:07.252006+0000","mapping_epoch":13,"log_start":"130'258411","ondisk_log_start":"130'258411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3316158647,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":267098,"num_read_kb":1053272,"num_write":267414,"num_write_kb":1069641,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"134'282384","reported_seq":557769,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:07.269753+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:22:07.269753+0000","last_peered":"2023-10-14T01:22:07.269753+0000","last_clean":"2023-10-14T01:22:07.269753+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:22:07.269753+0000","last_undegraded":"2023-10-14T01:22:07.269753+0000","last_fullsized":"2023-10-14T01:22:07.269753+0000","mapping_epoch":13,"log_start":"131'272308","ondisk_log_start":"131'272308","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3343159563,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":277507,"num_read_kb":1072345,"num_write":268432,"num_write_kb":1073327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"134'288795","reported_seq":573531,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:07.199560+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:22:07.199560+0000","last_peered":"2023-10-14T01:22:07.199560+0000","last_clean":"2023-10-14T01:22:07.199560+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:22:07.199560+0000","last_undegraded":"2023-10-14T01:22:07.199560+0000","last_fullsized":"2023-10-14T01:22:07.199560+0000","mapping_epoch":13,"log_start":"131'278712","ondisk_log_start":"131'278712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":996,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 367s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3514552875,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":285448,"num_read_kb":1121942,"num_write":282678,"num_write_kb":1130591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"134'305513","reported_seq":600625,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:07.213555+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:22:07.213555+0000","last_peered":"2023-10-14T01:22:07.213555+0000","last_clean":"2023-10-14T01:22:07.213555+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:22:07.213555+0000","last_undegraded":"2023-10-14T01:22:07.213555+0000","last_fullsized":"2023-10-14T01:22:07.213555+0000","mapping_epoch":13,"log_start":"131'295512","ondisk_log_start":"131'295512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1206,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 439s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3611460015,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":295890,"num_read_kb":1150191,"num_write":290797,"num_write_kb":1162870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"134'293833","reported_seq":587786,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:07.754604+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:22:07.754604+0000","last_peered":"2023-10-14T01:22:07.754604+0000","last_clean":"2023-10-14T01:22:07.754604+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:22:07.754604+0000","last_undegraded":"2023-10-14T01:22:07.754604+0000","last_fullsized":"2023-10-14T01:22:07.754604+0000","mapping_epoch":13,"log_start":"131'283812","ondisk_log_start":"131'283812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3557798359,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":294142,"num_read_kb":1136163,"num_write":286490,"num_write_kb":1145453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"134'308818","reported_seq":614303,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:06.849063+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:22:06.849063+0000","last_peered":"2023-10-14T01:22:06.849063+0000","last_clean":"2023-10-14T01:22:06.849063+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:22:06.849063+0000","last_undegraded":"2023-10-14T01:22:06.849063+0000","last_fullsized":"2023-10-14T01:22:06.849063+0000","mapping_epoch":13,"log_start":"131'298812","ondisk_log_start":"131'298812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3777233316,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":305973,"num_read_kb":1198088,"num_write":302438,"num_write_kb":1209578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"134'305653","reported_seq":606480,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:06.835671+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:22:06.835671+0000","last_peered":"2023-10-14T01:22:06.835671+0000","last_clean":"2023-10-14T01:22:06.835671+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:22:06.835671+0000","last_undegraded":"2023-10-14T01:22:06.835671+0000","last_fullsized":"2023-10-14T01:22:06.835671+0000","mapping_epoch":13,"log_start":"131'295615","ondisk_log_start":"131'295615","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3726037345,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":301372,"num_read_kb":1180402,"num_write":299270,"num_write_kb":1196890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"134'311131","reported_seq":609726,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:07.225083+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:22:07.225083+0000","last_peered":"2023-10-14T01:22:07.225083+0000","last_clean":"2023-10-14T01:22:07.225083+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:22:07.225083+0000","last_undegraded":"2023-10-14T01:22:07.225083+0000","last_fullsized":"2023-10-14T01:22:07.225083+0000","mapping_epoch":13,"log_start":"131'301116","ondisk_log_start":"131'301116","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":930,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 386s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3639615924,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":299237,"num_read_kb":1162971,"num_write":292385,"num_write_kb":1169183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":344,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:31.363006+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:21:31.363006+0000","last_peered":"2023-10-14T01:21:31.363006+0000","last_clean":"2023-10-14T01:21:31.363006+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:21:31.363006+0000","last_undegraded":"2023-10-14T01:21:31.363006+0000","last_fullsized":"2023-10-14T01:21:31.363006+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28486016044,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2326667,"num_read_kb":9075374,"num_write":2289904,"num_write_kb":9157533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18754957312,"data_stored":28631042136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80299,"ondisk_log_size":80299,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739354,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5174956,"kb_used_data":3576740,"kb_used_omap":484,"kb_used_meta":1597723,"kb_avail":89196884,"statfs":{"total":96636764160,"available":91337609216,"internally_reserved":0,"allocated":3662581760,"data_stored":5590912294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1636068676},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.182,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":3.9180000000000001,"5min":4.367,"15min":6.1120000000000001},"last":0.28000000000000003},{"interface":"front","average":{"1min":1.9570000000000001,"5min":1.3220000000000001,"15min":1.0569999999999999},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":11.481999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:21:41 2023","interfaces":[{"interface":"back","average":{"1min":1.8260000000000001,"5min":2.032,"15min":1.984},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":9.9900000000000002,"5min":44.308,"15min":44.308},"last":7.3760000000000003},{"interface":"front","average":{"1min":1.8899999999999999,"5min":2.0950000000000002,"15min":2.085},"min":{"1min":0.313,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":10.026,"5min":44.350000000000001,"15min":44.350000000000001},"last":7.4109999999999996}]}]},{"osd":1,"up_from":8,"seq":34359739345,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10421104,"kb_used_data":9157416,"kb_used_omap":1204,"kb_used_meta":1262475,"kb_avail":83950736,"statfs":{"total":96636764160,"available":85965553664,"internally_reserved":0,"allocated":9377193984,"data_stored":14315095737,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1292775149},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:21:14 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.70099999999999996,"15min":0.79600000000000004},"min":{"1min":0.20999999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":3.8690000000000002,"5min":7.2560000000000002,"15min":15.132999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.0609999999999999,"15min":0.88900000000000001},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":5.2690000000000001,"5min":8.4670000000000005,"15min":15.173},"last":0.33900000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":1.0309999999999999,"15min":0.97099999999999997},"min":{"1min":0.27400000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":3.8109999999999999,"5min":9.6020000000000003,"15min":10.403},"last":1.6819999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.78400000000000003,"15min":0.79700000000000004},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":4.069,"5min":7.6299999999999999,"15min":7.6299999999999999},"last":1.534}]}]},{"osd":0,"up_from":8,"seq":34359739356,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6778332,"kb_used_data":5583252,"kb_used_omap":718,"kb_used_meta":1194353,"kb_avail":87593508,"statfs":{"total":96636764160,"available":89695752192,"internally_reserved":0,"allocated":5717250048,"data_stored":8726147584,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1223018079},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:21:11 2023","interfaces":[{"interface":"back","average":{"1min":1.591,"5min":1.3180000000000001,"15min":0.85899999999999999},"min":{"1min":0.24099999999999999,"5min":0.20000000000000001,"15min":0.182},"max":{"1min":7.9459999999999997,"5min":19.274000000000001,"15min":19.274000000000001},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.875,"15min":0.745},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":3.6549999999999998,"5min":19.376000000000001,"15min":19.376000000000001},"last":0.28699999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:21:11 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.75800000000000001,"15min":0.92400000000000004},"min":{"1min":0.23499999999999999,"5min":0.216,"15min":0.20200000000000001},"max":{"1min":3.5579999999999998,"5min":9.3309999999999995,"15min":9.3309999999999995},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.75700000000000001,"15min":0.72499999999999998},"min":{"1min":0.23999999999999999,"5min":0.214,"15min":0.214},"max":{"1min":4.2439999999999998,"5min":7.5979999999999999,"15min":7.5979999999999999},"last":0.25600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5716221952,"data_stored":8725260147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9377308672,"data_stored":14315351084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3661426688,"data_stored":5590430905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:22:10.216 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:10.216 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:10.217 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:10.217 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:10.218 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:10.218 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:22:10.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:10.736+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:10.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:10.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:10.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:10.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:10.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:10.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:10.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:10.824+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:10.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:10.824+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:10.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:10.824+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:11.707 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:11.700+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:22:11.707 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:11.700+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:22:11.708 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:11.700+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:11.708 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:11.700+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:12.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.164+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:12.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:12.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:12.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:12.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.188+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:12.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.188+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:12.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.188+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:12.410 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:12.404+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:22:12.413 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:12.408+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:22:12.421 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:12.416+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:12.430 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:12.424+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:12.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:12.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:12.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.892+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:12.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.892+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:12.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.900+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:12.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:12.900+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:16.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:16.278 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:16.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:16.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:16.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.276+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:16.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:16.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:16.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:16.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:16.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.344+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:16.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:16.344+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:18.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:18.560+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:18.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:18.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:18.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:18.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:18.586 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:18.584+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:19.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:19.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:19.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:19.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:19.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:19.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:20.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.072+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:20.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.088+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:20.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:20.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:20.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.116+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:20.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.116+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:20.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.116+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:20.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:20.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.836+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:20.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:20.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.904+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:20.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.992+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:20.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:20.992+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:21.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:21.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:21.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:21.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.100+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:21.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.100+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:21.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.116+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:21.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.316+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:21.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:21.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:21.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:21.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:21.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.420+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:21.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.420+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:21.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:21.420+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:22.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.156+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:22.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:22.210 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:22.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:22.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:22.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.228+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:22.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.236+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:22.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:22.236+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:23.451 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:23.448+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:22:23.530 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:23.528+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:24.617 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:24.615+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:22:24.641 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:24.639+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:24.650 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:24.647+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:25.360 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:25.359+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:22:25.374 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:25.371+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:25.394 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:25.391+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:25.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.679+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:25.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:25.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:25.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:25.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:25.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:25.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.747+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:25.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.747+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:25.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.763+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:25.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.911+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:25.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:25.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.939+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:25.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:25.959+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:26.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:26.019+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:26.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:26.019+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:26.200 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:26.199+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:26.244 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:26.243+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:28.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:28.915+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:28.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:28.935+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:28.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:28.955+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:29.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.095+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:29.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:29.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.123+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:29.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:29.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.131+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:29.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:29.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:29.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:29.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.703+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:29.705 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.703+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:29.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.715+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:29.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:29.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:30.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:22:30.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:30.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:30.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:30.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:30.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:30.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:30.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:30.759+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:30.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:30.759+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:30.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:30.759+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:30.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:30.759+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:30.905 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:22:30.906 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:22:30.940 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2619,"stamp":"2023-10-14T01:22:30.856141+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28518870604,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2334416,"num_read_kb":9105907,"num_write":2297965,"num_write_kb":9191649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80569,"ondisk_log_size":80569,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":22670120,"kb_used_data":18380496,"kb_used_omap":2407,"kb_used_meta":4287192,"kb_avail":260445400,"statfs":{"total":289910292480,"available":266696089600,"internally_reserved":0,"allocated":18821627904,"data_stored":28696757727,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":4390085264},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,1,14,12,11,10,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16158720,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3753,"num_read_kb":14988,"num_write":3957,"num_write_kb":15828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007462"},"pg_stats":[{"pgid":"2.7","version":"134'269423","reported_seq":536544,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:28.043276+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:22:28.043276+0000","last_peered":"2023-10-14T01:22:28.043276+0000","last_clean":"2023-10-14T01:22:28.043276+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:22:28.043276+0000","last_undegraded":"2023-10-14T01:22:28.043276+0000","last_fullsized":"2023-10-14T01:22:28.043276+0000","mapping_epoch":13,"log_start":"131'259411","ondisk_log_start":"131'259411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3320074423,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":267964,"num_read_kb":1056724,"num_write":268367,"num_write_kb":1073453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"134'283401","reported_seq":559677,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:28.051432+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:22:28.051432+0000","last_peered":"2023-10-14T01:22:28.051432+0000","last_clean":"2023-10-14T01:22:28.051432+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:22:28.051432+0000","last_undegraded":"2023-10-14T01:22:28.051432+0000","last_fullsized":"2023-10-14T01:22:28.051432+0000","mapping_epoch":13,"log_start":"131'273308","ondisk_log_start":"131'273308","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3347083531,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":278388,"num_read_kb":1075853,"num_write":269395,"num_write_kb":1077179,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"134'289822","reported_seq":575489,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:28.050897+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:22:28.050897+0000","last_peered":"2023-10-14T01:22:28.050897+0000","last_clean":"2023-10-14T01:22:28.050897+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:22:28.050897+0000","last_undegraded":"2023-10-14T01:22:28.050897+0000","last_fullsized":"2023-10-14T01:22:28.050897+0000","mapping_epoch":13,"log_start":"131'279812","ondisk_log_start":"131'279812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1054,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 388s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3518558763,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":286369,"num_read_kb":1125614,"num_write":283646,"num_write_kb":1134463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"134'306611","reported_seq":602720,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:28.026235+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:22:28.026235+0000","last_peered":"2023-10-14T01:22:28.026235+0000","last_clean":"2023-10-14T01:22:28.026235+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:22:28.026235+0000","last_undegraded":"2023-10-14T01:22:28.026235+0000","last_fullsized":"2023-10-14T01:22:28.026235+0000","mapping_epoch":13,"log_start":"131'296512","ondisk_log_start":"131'296512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1244,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 460s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3615646127,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":296869,"num_read_kb":1154103,"num_write":291821,"num_write_kb":1166966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"134'294807","reported_seq":589752,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:28.013148+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:22:28.013148+0000","last_peered":"2023-10-14T01:22:28.013148+0000","last_clean":"2023-10-14T01:22:28.013148+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:22:28.013148+0000","last_undegraded":"2023-10-14T01:22:28.013148+0000","last_fullsized":"2023-10-14T01:22:28.013148+0000","mapping_epoch":13,"log_start":"131'284712","ondisk_log_start":"131'284712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3561705943,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":295111,"num_read_kb":1140023,"num_write":287449,"num_write_kb":1149289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"134'309845","reported_seq":616347,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:26.982703+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:22:26.982703+0000","last_peered":"2023-10-14T01:22:26.982703+0000","last_clean":"2023-10-14T01:22:26.982703+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:22:26.982703+0000","last_undegraded":"2023-10-14T01:22:26.982703+0000","last_fullsized":"2023-10-14T01:22:26.982703+0000","mapping_epoch":13,"log_start":"131'299812","ondisk_log_start":"131'299812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3781304740,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":306983,"num_read_kb":1202124,"num_write":303441,"num_write_kb":1213590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"134'306691","reported_seq":608516,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:26.992573+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:22:26.992573+0000","last_peered":"2023-10-14T01:22:26.992573+0000","last_clean":"2023-10-14T01:22:26.992573+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:22:26.992573+0000","last_undegraded":"2023-10-14T01:22:26.992573+0000","last_fullsized":"2023-10-14T01:22:26.992573+0000","mapping_epoch":13,"log_start":"131'296615","ondisk_log_start":"131'296615","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3730170209,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":302365,"num_read_kb":1184374,"num_write":300284,"num_write_kb":1200946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"134'312240","reported_seq":611863,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:28.007597+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:22:28.007597+0000","last_peered":"2023-10-14T01:22:28.007597+0000","last_clean":"2023-10-14T01:22:28.007597+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:22:28.007597+0000","last_undegraded":"2023-10-14T01:22:28.007597+0000","last_fullsized":"2023-10-14T01:22:28.007597+0000","mapping_epoch":13,"log_start":"131'302216","ondisk_log_start":"131'302216","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":977,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 407s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3643736500,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":300241,"num_read_kb":1166983,"num_write":293393,"num_write_kb":1173215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":344,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:31.363006+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:21:31.363006+0000","last_peered":"2023-10-14T01:21:31.363006+0000","last_clean":"2023-10-14T01:21:31.363006+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:21:31.363006+0000","last_undegraded":"2023-10-14T01:21:31.363006+0000","last_fullsized":"2023-10-14T01:21:31.363006+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28518280236,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2334290,"num_read_kb":9105798,"num_write":2297796,"num_write_kb":9189101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18819407872,"data_stored":28695492696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80442,"ondisk_log_size":80442,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739358,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5252840,"kb_used_data":3588000,"kb_used_omap":484,"kb_used_meta":1664347,"kb_avail":89119000,"statfs":{"total":96636764160,"available":91257856000,"internally_reserved":0,"allocated":3674112000,"data_stored":5602442534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1704291652},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,5,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:21:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.54900000000000004,"15min":0.71199999999999997},"min":{"1min":0.182,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":3.9180000000000001,"5min":4.367,"15min":6.1120000000000001},"last":0.23300000000000001},{"interface":"front","average":{"1min":1.9570000000000001,"5min":1.3220000000000001,"15min":1.0569999999999999},"min":{"1min":0.218,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":11.481999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.20599999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:21:41 2023","interfaces":[{"interface":"back","average":{"1min":1.8260000000000001,"5min":2.032,"15min":1.984},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":9.9900000000000002,"5min":44.308,"15min":44.308},"last":10.878},{"interface":"front","average":{"1min":1.8899999999999999,"5min":2.0950000000000002,"15min":2.085},"min":{"1min":0.313,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":10.026,"5min":44.350000000000001,"15min":44.350000000000001},"last":10.912000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739349,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10521012,"kb_used_data":9190444,"kb_used_omap":1204,"kb_used_meta":1329355,"kb_avail":83850828,"statfs":{"total":96636764160,"available":85863247872,"internally_reserved":0,"allocated":9411014656,"data_stored":14348916409,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1361260269},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,6,9,9,9,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68799999999999994,"15min":0.76300000000000001},"min":{"1min":0.23000000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":2.3239999999999998,"5min":7.2560000000000002,"15min":15.132999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.095,"15min":0.89700000000000002},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":3.6080000000000001,"5min":8.4670000000000005,"15min":15.173},"last":0.39800000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:22:22 2023","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.258,"15min":0.999},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.22500000000000001},"max":{"1min":8.2409999999999997,"5min":9.6020000000000003,"15min":9.6020000000000003},"last":0.42799999999999999},{"interface":"front","average":{"1min":1.337,"5min":0.81000000000000005,"15min":0.84199999999999997},"min":{"1min":0.23899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":8.202,"5min":8.202,"15min":8.202},"last":0.48599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739360,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":6896268,"kb_used_data":5602052,"kb_used_omap":718,"kb_used_meta":1293489,"kb_avail":87475572,"statfs":{"total":96636764160,"available":89574985728,"internally_reserved":0,"allocated":5736501248,"data_stored":8745398784,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1324533343},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:22:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2450000000000001,"15min":0.89700000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.183},"max":{"1min":9.4510000000000005,"5min":9.4510000000000005,"15min":19.274000000000001},"last":0.31},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.2169999999999996,"5min":5.2169999999999996,"15min":19.376000000000001},"last":0.28000000000000003}]},{"osd":2,"last update":"Sat Oct 14 01:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70899999999999996,"15min":0.92300000000000004},"min":{"1min":0.222,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":3.427,"5min":6.4530000000000003,"15min":9.3309999999999995},"last":1.1830000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78700000000000003,"15min":0.73899999999999999},"min":{"1min":0.249,"5min":0.214,"15min":0.214},"max":{"1min":5.1719999999999997,"5min":7.5979999999999999,"15min":7.5979999999999999},"last":1.212}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5735735296,"data_stored":8744773491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9410048000,"data_stored":14348090412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3673624576,"data_stored":5602628793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:22:30.942 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:30.943 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:30.943 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:30.944 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:30.944 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:30.945 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:22:31.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.387+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:31.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:31.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.403+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:31.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:31.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:31.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.423+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:31.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.431+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:31.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.431+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:31.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.519+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:22:31.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.519+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:22:31.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.519+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:31.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:31.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:31.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:31.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.555+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:31.838 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:22:31.921 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:22:31.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.923+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:31.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:31.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:31.995 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:31.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:31.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:31.991+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:32.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:32.007+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:32.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:32.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:32.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:32.243+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:32.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:32.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:32.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:32.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:32.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:32.255+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:32.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:32.255+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:32.820 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:22:32.858 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:22:33.736 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:22:33.803 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:22:33.997 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:33.991+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:34.007 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:34.007+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:34.517 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:22:34.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:34.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:34.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:34.557 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:22:34.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.575+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:34.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:34.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:34.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.655+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:34.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.859+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:34.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:34.886 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:34.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:34.935+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:35.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.219+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:35.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.227+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:35.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:35.266 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.259+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:35.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.403+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:35.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:35.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:35.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.455+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:35.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.455+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:35.537 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:22:35.546 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:35.543+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:35.561 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:35.559+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:35.602 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:22:35.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.911+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:35.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:35.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:35.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:35.947+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:39.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:39.587+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:39.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:39.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:39.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:39.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:39.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:39.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:39.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:39.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:39.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:39.683+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:40.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:40.467+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:40.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:40.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:40.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:40.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:40.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:40.503+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:40.516 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:40.511+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:40.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:40.559+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:40.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:40.575+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:46.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:46.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:46.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:46.807+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:46.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:46.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:46.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:46.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:46.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:46.811+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:46.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:46.811+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:46.818 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:46.815+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:48.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:48.919+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:48.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:48.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:48.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:48.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:48.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:48.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:48.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:48.963+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:48.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:48.963+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:48.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:48.963+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:52.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.575+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:52.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:52.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:52.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:52.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:52.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.707+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:52.709 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.707+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:52.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.723+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:52.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.751+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:22:52.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:52.771 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:52.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:52.771+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:52.807 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:52.807+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:22:52.813 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:52.811+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:52.813 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:52.811+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:53.951 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:53.951+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:55.604 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:22:55.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.875+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:22:55.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:22:55.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.959+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:22:55.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:55.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:22:55.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.967+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:22:55.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.967+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:55.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:55.971+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:22:56.371 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:22:56.372 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:22:56.430 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2631,"stamp":"2023-10-14T01:22:54.862049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28560150092,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2344146,"num_read_kb":9144691,"num_write":2308021,"num_write_kb":9231873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80609,"ondisk_log_size":80609,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23079396,"kb_used_data":18460812,"kb_used_omap":2407,"kb_used_meta":4616152,"kb_avail":260036124,"statfs":{"total":289910292480,"available":266276990976,"internally_reserved":0,"allocated":18903871488,"data_stored":28779001311,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":4726940304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,8,5,3,11,11,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23248896,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5456,"num_read_kb":21764,"num_write":5657,"num_write_kb":22628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002124"},"pg_stats":[{"pgid":"2.7","version":"134'270582","reported_seq":538848,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:53.837254+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:22:53.837254+0000","last_peered":"2023-10-14T01:22:53.837254+0000","last_clean":"2023-10-14T01:22:53.837254+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:22:53.837254+0000","last_undegraded":"2023-10-14T01:22:53.837254+0000","last_fullsized":"2023-10-14T01:22:53.837254+0000","mapping_epoch":13,"log_start":"131'260511","ondisk_log_start":"131'260511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3324866743,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":269107,"num_read_kb":1061276,"num_write":269526,"num_write_kb":1078089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"134'284644","reported_seq":562107,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:53.849490+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:22:53.849490+0000","last_peered":"2023-10-14T01:22:53.849490+0000","last_clean":"2023-10-14T01:22:53.849490+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:22:53.849490+0000","last_undegraded":"2023-10-14T01:22:53.849490+0000","last_fullsized":"2023-10-14T01:22:53.849490+0000","mapping_epoch":13,"log_start":"131'274608","ondisk_log_start":"131'274608","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3351892235,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":279555,"num_read_kb":1080521,"num_write":270572,"num_write_kb":1081887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"134'291111","reported_seq":577960,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:53.759267+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:22:53.759267+0000","last_peered":"2023-10-14T01:22:53.759267+0000","last_clean":"2023-10-14T01:22:53.759267+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:22:53.759267+0000","last_undegraded":"2023-10-14T01:22:53.759267+0000","last_fullsized":"2023-10-14T01:22:53.759267+0000","mapping_epoch":13,"log_start":"131'281013","ondisk_log_start":"131'281013","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1144,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 414s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3523514923,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":287532,"num_read_kb":1130246,"num_write":284854,"num_write_kb":1139295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"134'308043","reported_seq":605401,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:53.796992+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:22:53.796992+0000","last_peered":"2023-10-14T01:22:53.796992+0000","last_clean":"2023-10-14T01:22:53.796992+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:22:53.796992+0000","last_undegraded":"2023-10-14T01:22:53.796992+0000","last_fullsized":"2023-10-14T01:22:53.796992+0000","mapping_epoch":13,"log_start":"132'298013","ondisk_log_start":"132'298013","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1297,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 486s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3621122479,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":298094,"num_read_kb":1158991,"num_write":293153,"num_write_kb":1172294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"134'296081","reported_seq":592284,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:53.250853+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:22:53.250853+0000","last_peered":"2023-10-14T01:22:53.250853+0000","last_clean":"2023-10-14T01:22:53.250853+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:22:53.250853+0000","last_undegraded":"2023-10-14T01:22:53.250853+0000","last_fullsized":"2023-10-14T01:22:53.250853+0000","mapping_epoch":13,"log_start":"131'286012","ondisk_log_start":"131'286012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3566838231,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":296349,"num_read_kb":1144955,"num_write":288703,"num_write_kb":1154305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"134'311135","reported_seq":618945,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:52.227832+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:22:52.227832+0000","last_peered":"2023-10-14T01:22:52.227832+0000","last_clean":"2023-10-14T01:22:52.227832+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:22:52.227832+0000","last_undegraded":"2023-10-14T01:22:52.227832+0000","last_fullsized":"2023-10-14T01:22:52.227832+0000","mapping_epoch":13,"log_start":"131'301112","ondisk_log_start":"131'301112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3786580388,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":308281,"num_read_kb":1207288,"num_write":304725,"num_write_kb":1218726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"134'307997","reported_seq":611159,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:22:52.211265+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:22:52.211265+0000","last_peered":"2023-10-14T01:22:52.211265+0000","last_clean":"2023-10-14T01:22:52.211265+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:22:52.211265+0000","last_undegraded":"2023-10-14T01:22:52.211265+0000","last_fullsized":"2023-10-14T01:22:52.211265+0000","mapping_epoch":13,"log_start":"131'297915","ondisk_log_start":"131'297915","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3735486817,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":303691,"num_read_kb":1189662,"num_write":301584,"num_write_kb":1206146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"134'313689","reported_seq":614513,"reported_epoch":134,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:22:53.788085+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:22:53.788085+0000","last_peered":"2023-10-14T01:22:53.788085+0000","last_clean":"2023-10-14T01:22:53.788085+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:22:53.788085+0000","last_undegraded":"2023-10-14T01:22:53.788085+0000","last_fullsized":"2023-10-14T01:22:53.788085+0000","mapping_epoch":13,"log_start":"132'303616","ondisk_log_start":"132'303616","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1016,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 432s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3649257908,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":301411,"num_read_kb":1171643,"num_write":294735,"num_write_kb":1178583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":344,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-10-14T01:21:31.363006+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:21:31.363006+0000","last_peered":"2023-10-14T01:21:31.363006+0000","last_clean":"2023-10-14T01:21:31.363006+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:21:31.363006+0000","last_undegraded":"2023-10-14T01:21:31.363006+0000","last_fullsized":"2023-10-14T01:21:31.363006+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28559559724,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2344020,"num_read_kb":9144582,"num_write":2307852,"num_write_kb":9229325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18901663744,"data_stored":28777748568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80482,"ondisk_log_size":80482,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739363,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5323272,"kb_used_data":3604288,"kb_used_omap":484,"kb_used_meta":1718491,"kb_avail":89048568,"statfs":{"total":96636764160,"available":91185733632,"internally_reserved":0,"allocated":3690790912,"data_stored":5619121446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1759735108},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55300000000000005,"15min":0.69599999999999995},"min":{"1min":0.20100000000000001,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":2.5009999999999999,"5min":4.367,"15min":6.1120000000000001},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":1.1459999999999999,"15min":1.054},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.5449999999999999,"5min":11.481999999999999,"15min":11.554},"last":4.6109999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:22:43 2023","interfaces":[{"interface":"back","average":{"1min":2.367,"5min":2.125,"15min":2.028},"min":{"1min":0.245,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":10.878,"5min":44.308,"15min":44.308},"last":4.3920000000000003},{"interface":"front","average":{"1min":2.4089999999999998,"5min":2.1749999999999998,"15min":2.1269999999999998},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":10.912000000000001,"5min":44.350000000000001,"15min":44.350000000000001},"last":4.4290000000000003}]}]},{"osd":1,"up_from":8,"seq":34359739354,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10753712,"kb_used_data":9230248,"kb_used_omap":1204,"kb_used_meta":1522251,"kb_avail":83618128,"statfs":{"total":96636764160,"available":85624963072,"internally_reserved":0,"allocated":9451773952,"data_stored":14389675705,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1558785773},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,2,1,11,11,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68799999999999994,"15min":0.76300000000000001},"min":{"1min":0.23000000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":2.3239999999999998,"5min":7.2560000000000002,"15min":15.132999999999999},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.095,"15min":0.89700000000000002},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":3.6080000000000001,"5min":8.4670000000000005,"15min":15.173},"last":0.30599999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:22:22 2023","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.258,"15min":0.999},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.22500000000000001},"max":{"1min":8.2409999999999997,"5min":9.6020000000000003,"15min":9.6020000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":1.337,"5min":0.81000000000000005,"15min":0.84199999999999997},"min":{"1min":0.23899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":8.202,"5min":8.202,"15min":8.202},"last":0.32700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739365,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7002412,"kb_used_data":5626276,"kb_used_omap":718,"kb_used_meta":1375409,"kb_avail":87369428,"statfs":{"total":96636764160,"available":89466294272,"internally_reserved":0,"allocated":5761306624,"data_stored":8770204160,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1408419423},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:22:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2450000000000001,"15min":0.89700000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.183},"max":{"1min":9.4510000000000005,"5min":9.4510000000000005,"15min":19.274000000000001},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.2169999999999996,"5min":5.2169999999999996,"15min":19.376000000000001},"last":0.22}]},{"osd":2,"last update":"Sat Oct 14 01:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70899999999999996,"15min":0.92300000000000004},"min":{"1min":0.222,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":3.427,"5min":6.4530000000000003,"15min":9.3309999999999995},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78700000000000003,"15min":0.73899999999999999},"min":{"1min":0.249,"5min":0.214,"15min":0.214},"max":{"1min":5.1719999999999997,"5min":7.5979999999999999,"15min":7.5979999999999999},"last":0.377}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5760401408,"data_stored":8769439603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9451622400,"data_stored":14389664812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3689639936,"data_stored":5618644153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:22:56.431 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:56.432 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:56.432 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:56.433 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:56.433 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:22:56.433 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:22:56.792 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:56.791+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:22:58.642 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:22:58.639+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:22:59.396 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:59.395+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:22:59.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:59.435+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:22:59.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:22:59.439+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:00.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:00.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:00.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:00.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:00.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:00.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:00.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:00.683+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:00.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:00.683+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:00.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:00.683+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:01.208 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:01.207+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:01.216 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:01.215+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:01.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:01.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.391+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:01.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:01.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.427+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:01.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.427+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:01.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.435+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:01.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.435+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:01.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:01.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:01.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:01.689 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:01.687+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:23:02.048 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:02.047+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:02.054 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:02.051+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:23:02.070 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:02.067+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:02.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:02.619+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:02.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:02.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:02.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:02.679+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:03.985 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:03.983+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:23:03.995 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:03.991+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:06.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.818+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:06.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:06.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:06.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:06.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.918+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:06.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.918+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:06.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.918+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:06.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:06.942+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:07.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.198+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:07.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.214+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:07.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:07.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.234+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:07.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.262+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:07.267 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.262+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:07.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.266+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:07.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:07.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.274+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:07.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:07.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:07.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.430+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:07.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:07.430+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:08.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:08.982+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:09.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:09.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:09.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:09.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.046+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:09.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.794+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:09.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.810+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:09.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.854+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:09.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:09.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.874+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:09.882+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:10.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.078+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:10.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:10.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:10.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:10.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.158+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:10.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.318+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:10.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.334+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:10.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.342+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:10.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:10.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.350+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:10.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.830+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:10.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:10.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:10.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:10.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:10.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.902+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:10.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:10.914+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:12.734 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:12.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:12.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:12.742+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:12.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:12.750+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:13.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:13.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:13.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:13.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.146+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:13.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.190+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:13.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:13.204 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.202+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:13.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.626+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:13.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:13.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:13.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:13.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:13.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:13.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:13.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:13.682+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:14.057 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:14.054+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:14.057 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:14.054+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:15.219 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:15.218+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:16.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.322+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:16.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:16.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:16.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:16.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:16.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.366+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:16.375 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.366+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:16.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:16.370+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:16.435 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:23:17.167 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:23:17.168 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:23:17.239 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2643,"stamp":"2023-10-14T01:23:16.865683+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28592545356,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2352042,"num_read_kb":9176187,"num_write":2315931,"num_write_kb":9263513,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80451,"ondisk_log_size":80451,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23265864,"kb_used_data":18524076,"kb_used_omap":2407,"kb_used_meta":4739352,"kb_avail":259849656,"statfs":{"total":289910292480,"available":266086047744,"internally_reserved":0,"allocated":18968653824,"data_stored":28843770144,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":4853097104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,9,10,13,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15990784,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3842,"num_read_kb":15340,"num_write":3906,"num_write_kb":15624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001669"},"pg_stats":[{"pgid":"2.7","version":"135'271517","reported_seq":540731,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:14.596544+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:23:14.596544+0000","last_peered":"2023-10-14T01:23:14.596544+0000","last_clean":"2023-10-14T01:23:14.596544+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:23:14.596544+0000","last_undegraded":"2023-10-14T01:23:14.596544+0000","last_fullsized":"2023-10-14T01:23:14.596544+0000","mapping_epoch":13,"log_start":"131'261511","ondisk_log_start":"131'261511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3328704695,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":270054,"num_read_kb":1065056,"num_write":270461,"num_write_kb":1081829,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"135'285629","reported_seq":564050,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:14.569753+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:23:14.569753+0000","last_peered":"2023-10-14T01:23:14.569753+0000","last_clean":"2023-10-14T01:23:14.569753+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:23:14.569753+0000","last_undegraded":"2023-10-14T01:23:14.569753+0000","last_fullsized":"2023-10-14T01:23:14.569753+0000","mapping_epoch":13,"log_start":"132'275608","ondisk_log_start":"132'275608","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3355681035,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":280500,"num_read_kb":1084301,"num_write":271498,"num_write_kb":1085591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"135'292176","reported_seq":579991,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:14.624592+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:23:14.624592+0000","last_peered":"2023-10-14T01:23:14.624592+0000","last_clean":"2023-10-14T01:23:14.624592+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:23:14.624592+0000","last_undegraded":"2023-10-14T01:23:14.624592+0000","last_fullsized":"2023-10-14T01:23:14.624592+0000","mapping_epoch":13,"log_start":"132'282113","ondisk_log_start":"132'282113","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1179,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 434s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3527692843,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":288485,"num_read_kb":1134042,"num_write":285877,"num_write_kb":1143387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"135'309145","reported_seq":607557,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:14.497106+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:23:14.497106+0000","last_peered":"2023-10-14T01:23:14.497106+0000","last_clean":"2023-10-14T01:23:14.497106+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:23:14.497106+0000","last_undegraded":"2023-10-14T01:23:14.497106+0000","last_fullsized":"2023-10-14T01:23:14.497106+0000","mapping_epoch":13,"log_start":"132'299113","ondisk_log_start":"132'299113","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1352,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 507s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3625284015,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":299126,"num_read_kb":1163107,"num_write":294172,"num_write_kb":1176370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"135'297073","reported_seq":594282,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:13.529875+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:23:13.529875+0000","last_peered":"2023-10-14T01:23:13.529875+0000","last_clean":"2023-10-14T01:23:13.529875+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:23:13.529875+0000","last_undegraded":"2023-10-14T01:23:13.529875+0000","last_fullsized":"2023-10-14T01:23:13.529875+0000","mapping_epoch":13,"log_start":"132'287012","ondisk_log_start":"132'287012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3570868695,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":297334,"num_read_kb":1148879,"num_write":289680,"num_write_kb":1158213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"135'312142","reported_seq":620972,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:12.301375+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:23:12.301375+0000","last_peered":"2023-10-14T01:23:12.301375+0000","last_clean":"2023-10-14T01:23:12.301375+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:23:12.301375+0000","last_undegraded":"2023-10-14T01:23:12.301375+0000","last_fullsized":"2023-10-14T01:23:12.301375+0000","mapping_epoch":13,"log_start":"132'302112","ondisk_log_start":"132'302112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3790631332,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":309294,"num_read_kb":1211324,"num_write":305714,"num_write_kb":1222682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"135'309063","reported_seq":613254,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:12.215406+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:23:12.215406+0000","last_peered":"2023-10-14T01:23:12.215406+0000","last_clean":"2023-10-14T01:23:12.215406+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:23:12.215406+0000","last_undegraded":"2023-10-14T01:23:12.215406+0000","last_fullsized":"2023-10-14T01:23:12.215406+0000","mapping_epoch":13,"log_start":"132'299015","ondisk_log_start":"132'299015","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3739763041,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":304711,"num_read_kb":1193734,"num_write":302633,"num_write_kb":1210342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"135'314779","reported_seq":616631,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:14.453684+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:23:14.453684+0000","last_peered":"2023-10-14T01:23:14.453684+0000","last_clean":"2023-10-14T01:23:14.453684+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:23:14.453684+0000","last_undegraded":"2023-10-14T01:23:14.453684+0000","last_fullsized":"2023-10-14T01:23:14.453684+0000","mapping_epoch":13,"log_start":"132'304716","ondisk_log_start":"132'304716","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1066,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 453s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3653329332,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":302412,"num_read_kb":1175635,"num_write":295727,"num_write_kb":1182551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":345,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-14T01:21:58.696441+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:21:58.696441+0000","last_peered":"2023-10-14T01:21:58.696441+0000","last_clean":"2023-10-14T01:21:58.696441+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:21:58.696441+0000","last_undegraded":"2023-10-14T01:21:58.696441+0000","last_fullsized":"2023-10-14T01:21:58.696441+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28591954988,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2351916,"num_read_kb":9176078,"num_write":2315762,"num_write_kb":9260965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18966388736,"data_stored":28842473560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80324,"ondisk_log_size":80324,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739367,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5366416,"kb_used_data":3615752,"kb_used_omap":484,"kb_used_meta":1750171,"kb_avail":89005424,"statfs":{"total":96636764160,"available":91141554176,"internally_reserved":0,"allocated":3702530048,"data_stored":5630856081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1792175428},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55300000000000005,"15min":0.69599999999999995},"min":{"1min":0.20100000000000001,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":2.5009999999999999,"5min":4.367,"15min":6.1120000000000001},"last":2.1379999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":1.1459999999999999,"15min":1.054},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.5449999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.224}]},{"osd":1,"last update":"Sat Oct 14 01:22:43 2023","interfaces":[{"interface":"back","average":{"1min":2.367,"5min":2.125,"15min":2.028},"min":{"1min":0.245,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":10.878,"5min":44.308,"15min":44.308},"last":2.9860000000000002},{"interface":"front","average":{"1min":2.4089999999999998,"5min":2.1749999999999998,"15min":2.1269999999999998},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":10.912000000000001,"5min":44.350000000000001,"15min":44.350000000000001},"last":3.0339999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739358,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10805872,"kb_used_data":9263076,"kb_used_omap":1204,"kb_used_meta":1541579,"kb_avail":83565968,"statfs":{"total":96636764160,"available":85571551232,"internally_reserved":0,"allocated":9485389824,"data_stored":14423287076,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1578577645},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,7,9,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:22:17 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.68799999999999994,"15min":0.76300000000000001},"min":{"1min":0.23000000000000001,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":2.3239999999999998,"5min":7.2560000000000002,"15min":15.132999999999999},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.86899999999999999,"5min":1.095,"15min":0.89700000000000002},"min":{"1min":0.27800000000000002,"5min":0.215,"15min":0.215},"max":{"1min":3.6080000000000001,"5min":8.4670000000000005,"15min":15.173},"last":0.26700000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:22:22 2023","interfaces":[{"interface":"back","average":{"1min":1.9730000000000001,"5min":1.258,"15min":0.999},"min":{"1min":0.33100000000000002,"5min":0.253,"15min":0.22500000000000001},"max":{"1min":8.2409999999999997,"5min":9.6020000000000003,"15min":9.6020000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":1.337,"5min":0.81000000000000005,"15min":0.84199999999999997},"min":{"1min":0.23899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":8.202,"5min":8.202,"15min":8.202},"last":1.546}]}]},{"osd":0,"up_from":8,"seq":34359739369,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7093576,"kb_used_data":5645248,"kb_used_omap":718,"kb_used_meta":1447601,"kb_avail":87278264,"statfs":{"total":96636764160,"available":89372942336,"internally_reserved":0,"allocated":5780733952,"data_stored":8789626987,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1482344031},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,2,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:22:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.2450000000000001,"15min":0.89700000000000002},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.183},"max":{"1min":9.4510000000000005,"5min":9.4510000000000005,"15min":19.274000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.22700000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.2169999999999996,"5min":5.2169999999999996,"15min":19.376000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:22:12 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.70899999999999996,"15min":0.92300000000000004},"min":{"1min":0.222,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":3.427,"5min":6.4530000000000003,"15min":9.3309999999999995},"last":0.44},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78700000000000003,"15min":0.73899999999999999},"min":{"1min":0.249,"5min":0.214,"15min":0.214},"max":{"1min":5.1719999999999997,"5min":7.5979999999999999,"15min":7.5979999999999999},"last":0.375}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5780029440,"data_stored":8789067635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9484414976,"data_stored":14422457388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3701944320,"data_stored":5630948537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:23:17.241 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:17.241 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:17.242 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:17.242 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:17.243 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:17.243 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:23:19.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.866+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:23:19.871 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.866+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:23:19.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.890+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:19.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:19.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:19.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:19.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:19.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.934+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:19.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.934+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:19.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.934+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:19.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:19.934+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:20.512 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:20.510+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:20.557 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:20.554+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:23:20.580 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:20.574+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:23.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.798+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:23.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:23.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:23.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:23.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:23.870 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.866+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:23.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.870+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:23.879 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:23.878+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:24.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.658+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:24.701 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:24.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.698+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:24.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:24.728 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.722+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:24.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.730+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:24.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.730+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:24.733 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:24.730+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:24.966 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:24.962+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:24.971 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:24.970+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:23:25.002 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:24.998+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:25.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:25.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:25.521 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:25.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:25.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:25.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:25.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:25.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:25.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:25.618+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:25.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:25.618+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:25.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:25.626+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:26.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:26.226+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:26.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:26.230+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:26.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:26.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:27.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.546+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:27.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.566+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:27.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:27.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.610+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:27.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.750+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:27.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:27.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.782+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:27.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:27.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:27.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:27.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.786+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:27.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.786+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:27.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:27.786+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:29.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:29.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:29.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:29.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:29.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:29.290+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:29.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:29.290+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:29.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:29.290+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:29.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:29.290+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:30.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.454+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:30.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:30.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:30.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:30.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:30.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.486+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:30.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.486+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:30.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:30.486+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:32.310 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:32.306+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:32.331 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:32.326+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:33.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:33.158+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:33.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:33.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:33.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:33.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:33.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:33.234+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:34.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:34.154+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:34.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:34.178+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:34.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:34.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:34.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:34.254+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:35.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.282+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:35.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.282+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:35.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:35.300 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:35.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:35.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.298+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:35.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.298+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:35.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.298+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:35.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.314+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:35.372 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.366+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:35.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:35.366+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:36.150 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:36.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:36.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:36.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:36.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:36.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:36.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:36.150+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:36.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:36.150+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:36.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:36.170+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:37.116 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:37.114+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:23:37.125 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:37.122+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:37.245 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:23:37.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:37.926+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:37.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:37.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:37.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:37.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:37.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:37.958+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:37.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:37.966+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:38.072 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:23:38.073 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:23:38.118 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2653,"stamp":"2023-10-14T01:23:36.872953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28625690188,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2360036,"num_read_kb":9208115,"num_write":2324015,"num_write_kb":9295849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80479,"ondisk_log_size":80479,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23646388,"kb_used_data":18587736,"kb_used_omap":2407,"kb_used_meta":5056216,"kb_avail":259469132,"statfs":{"total":289910292480,"available":265696391168,"internally_reserved":0,"allocated":19033841664,"data_stored":28908957984,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":5177565840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,1,7,5,16,9,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21487616,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5195,"num_read_kb":20756,"num_write":5246,"num_write_kb":20984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002129"},"pg_stats":[{"pgid":"2.7","version":"135'272441","reported_seq":542545,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:35.231491+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:23:35.231491+0000","last_peered":"2023-10-14T01:23:35.231491+0000","last_clean":"2023-10-14T01:23:35.231491+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:23:35.231491+0000","last_undegraded":"2023-10-14T01:23:35.231491+0000","last_fullsized":"2023-10-14T01:23:35.231491+0000","mapping_epoch":13,"log_start":"132'262411","ondisk_log_start":"132'262411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3332518071,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":270945,"num_read_kb":1068620,"num_write":271385,"num_write_kb":1085525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"135'286643","reported_seq":566033,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:35.210695+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:23:35.210695+0000","last_peered":"2023-10-14T01:23:35.210695+0000","last_clean":"2023-10-14T01:23:35.210695+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:23:35.210695+0000","last_undegraded":"2023-10-14T01:23:35.210695+0000","last_fullsized":"2023-10-14T01:23:35.210695+0000","mapping_epoch":13,"log_start":"132'276608","ondisk_log_start":"132'276608","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3359678731,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":281457,"num_read_kb":1088121,"num_write":272476,"num_write_kb":1089503,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"135'293228","reported_seq":582035,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:35.229606+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:23:35.229606+0000","last_peered":"2023-10-14T01:23:35.229606+0000","last_clean":"2023-10-14T01:23:35.229606+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:23:35.229606+0000","last_undegraded":"2023-10-14T01:23:35.229606+0000","last_fullsized":"2023-10-14T01:23:35.229606+0000","mapping_epoch":13,"log_start":"132'283213","ondisk_log_start":"132'283213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1220,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 455s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3531850283,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":289473,"num_read_kb":1137990,"num_write":286891,"num_write_kb":1147443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"135'310250","reported_seq":609649,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:35.188629+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:23:35.188629+0000","last_peered":"2023-10-14T01:23:35.188629+0000","last_clean":"2023-10-14T01:23:35.188629+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:23:35.188629+0000","last_undegraded":"2023-10-14T01:23:35.188629+0000","last_fullsized":"2023-10-14T01:23:35.188629+0000","mapping_epoch":13,"log_start":"132'300213","ondisk_log_start":"132'300213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1382,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 527s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3629531567,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":300099,"num_read_kb":1166995,"num_write":295214,"num_write_kb":1180538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"135'298067","reported_seq":596320,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:33.693549+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:23:33.693549+0000","last_peered":"2023-10-14T01:23:33.693549+0000","last_clean":"2023-10-14T01:23:33.693549+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:23:33.693549+0000","last_undegraded":"2023-10-14T01:23:33.693549+0000","last_fullsized":"2023-10-14T01:23:33.693549+0000","mapping_epoch":13,"log_start":"132'288012","ondisk_log_start":"132'288012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3574915543,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":298362,"num_read_kb":1152979,"num_write":290664,"num_write_kb":1162149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"135'313179","reported_seq":623059,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:32.520861+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:23:32.520861+0000","last_peered":"2023-10-14T01:23:32.520861+0000","last_clean":"2023-10-14T01:23:32.520861+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:23:32.520861+0000","last_undegraded":"2023-10-14T01:23:32.520861+0000","last_fullsized":"2023-10-14T01:23:32.520861+0000","mapping_epoch":13,"log_start":"132'303112","ondisk_log_start":"132'303112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3794780580,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":310336,"num_read_kb":1215492,"num_write":306737,"num_write_kb":1226774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"135'310157","reported_seq":615441,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:32.489233+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:23:32.489233+0000","last_peered":"2023-10-14T01:23:32.489233+0000","last_clean":"2023-10-14T01:23:32.489233+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:23:32.489233+0000","last_undegraded":"2023-10-14T01:23:32.489233+0000","last_fullsized":"2023-10-14T01:23:32.489233+0000","mapping_epoch":13,"log_start":"132'300115","ondisk_log_start":"132'300115","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3744207201,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":305798,"num_read_kb":1198082,"num_write":303713,"num_write_kb":1214662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"135'315887","reported_seq":618788,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:35.154320+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:23:35.154320+0000","last_peered":"2023-10-14T01:23:35.154320+0000","last_clean":"2023-10-14T01:23:35.154320+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:23:35.154320+0000","last_undegraded":"2023-10-14T01:23:35.154320+0000","last_fullsized":"2023-10-14T01:23:35.154320+0000","mapping_epoch":13,"log_start":"132'305816","ondisk_log_start":"132'305816","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1096,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 474s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3657617844,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":303440,"num_read_kb":1179727,"num_write":296766,"num_write_kb":1186707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":346,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:08.722459+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:23:08.722459+0000","last_peered":"2023-10-14T01:23:08.722459+0000","last_clean":"2023-10-14T01:23:08.722459+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:23:08.722459+0000","last_undegraded":"2023-10-14T01:23:08.722459+0000","last_fullsized":"2023-10-14T01:23:08.722459+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28625099820,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2359910,"num_read_kb":9208006,"num_write":2323846,"num_write_kb":9293301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19032297472,"data_stored":28908382296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80352,"ondisk_log_size":80352,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739371,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5429224,"kb_used_data":3628576,"kb_used_omap":484,"kb_used_meta":1800155,"kb_avail":88942616,"statfs":{"total":96636764160,"available":91077238784,"internally_reserved":0,"allocated":3715661824,"data_stored":5643987857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1843359044},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:22:43 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55300000000000005,"15min":0.69599999999999995},"min":{"1min":0.20100000000000001,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":2.5009999999999999,"5min":4.367,"15min":6.1120000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":1.1459999999999999,"15min":1.054},"min":{"1min":0.20599999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.5449999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.36399999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:22:43 2023","interfaces":[{"interface":"back","average":{"1min":2.367,"5min":2.125,"15min":2.028},"min":{"1min":0.245,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":10.878,"5min":44.308,"15min":44.308},"last":0.40000000000000002},{"interface":"front","average":{"1min":2.4089999999999998,"5min":2.1749999999999998,"15min":2.1269999999999998},"min":{"1min":0.27900000000000003,"5min":0.27700000000000002,"15min":0.27600000000000002},"max":{"1min":10.912000000000001,"5min":44.350000000000001,"15min":44.350000000000001},"last":0.435}]}]},{"osd":1,"up_from":8,"seq":34359739362,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11028096,"kb_used_data":9294772,"kb_used_omap":1204,"kb_used_meta":1732107,"kb_avail":83343744,"statfs":{"total":96636764160,"available":85343993856,"internally_reserved":0,"allocated":9517846528,"data_stored":14455743780,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1773678317},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,0,4,4,11,8,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.53200000000000003,"15min":0.76000000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":6.4020000000000001,"15min":15.132999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.0209999999999999,"15min":0.90100000000000002},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.215},"max":{"1min":3.6219999999999999,"5min":8.4670000000000005,"15min":15.173},"last":0.26100000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.165,"15min":1.006},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":2.2839999999999998,"5min":9.6020000000000003,"15min":9.6020000000000003},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.86299999999999999,"15min":0.84399999999999997},"min":{"1min":0.25700000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.774,"5min":8.202,"15min":8.202},"last":0.35099999999999998}]}]},{"osd":0,"up_from":8,"seq":34359739373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7189068,"kb_used_data":5664388,"kb_used_omap":718,"kb_used_meta":1523953,"kb_avail":87182772,"statfs":{"total":96636764160,"available":89275158528,"internally_reserved":0,"allocated":5800333312,"data_stored":8809226347,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1560528479},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:23:14 2023","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.4650000000000001,"15min":0.996},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":10.897,"5min":10.897,"15min":19.274000000000001},"last":0.41199999999999998},{"interface":"front","average":{"1min":1.383,"5min":0.86199999999999999,"15min":0.82999999999999996},"min":{"1min":0.22,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":10.669,"5min":10.669,"15min":19.376000000000001},"last":0.21199999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.68400000000000005,"15min":0.90100000000000002},"min":{"1min":0.23200000000000001,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":7.0739999999999998,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.25},{"interface":"front","average":{"1min":1.012,"5min":0.78100000000000003,"15min":0.77000000000000002},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.26700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5799874560,"data_stored":8808912755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9517744128,"data_stored":14455786540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3714678784,"data_stored":5643683001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:23:38.120 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:38.120 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:38.120 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:38.121 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:38.121 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:38.122 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:23:38.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:38.354+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:38.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:38.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:38.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:38.402+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:38.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:38.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:38.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:38.410+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:38.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:38.410+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:39.861 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:39.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:39.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:39.874+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:39.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:39.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:39.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:39.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:39.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:39.926+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:39.929 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:39.926+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:39.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:39.938+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:40.432 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:40.426+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:41.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:41.802+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:41.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:41.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:41.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:41.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:41.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:41.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:41.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:41.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:41.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:41.830+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:42.996 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:42.990+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:43.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:43.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:43.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:43.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:43.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:43.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:43.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:43.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:43.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:43.054+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:43.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:43.054+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:43.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:43.054+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:45.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:45.521+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:45.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:45.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:45.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:45.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:45.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:45.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:45.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:45.617+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:45.919 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:45.913+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:46.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:46.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.113+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:46.452 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:23:46.445+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:23:46.452 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:23:46.445+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:23:46.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:23:46.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:46.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:46.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.589+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:46.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.589+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:46.596 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.593+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:46.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.593+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:46.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.625+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:46.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:46.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:46.665+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:47.034 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:47.029+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:47.074 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:47.069+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:47.595 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:47.589+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:48.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:48.397+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:23:48.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:48.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:48.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:48.440 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:48.855 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:48.849+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:23:48.858 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:23:48.853+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:23:49.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:49.701+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:49.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:49.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:49.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:49.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:49.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:49.721+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:49.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:49.721+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:49.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:49.725+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:50.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:50.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:51.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:51.021+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:52.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:52.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:52.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:52.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:23:52.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:52.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:23:52.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:52.633+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:23:52.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:52.633+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:23:52.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:52.633+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:54.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:54.637+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:23:54.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:54.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:23:54.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:23:54.705+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:23:58.123 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:23:58.974 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:23:58.974 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:23:59.026 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2664,"stamp":"2023-10-14T01:23:58.877457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28660633164,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2368538,"num_read_kb":9242051,"num_write":2332571,"num_write_kb":9330073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80560,"ondisk_log_size":80560,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23957280,"kb_used_data":18655112,"kb_used_omap":2407,"kb_used_meta":5299736,"kb_avail":259158240,"statfs":{"total":289910292480,"available":265378037760,"internally_reserved":0,"allocated":19102834688,"data_stored":28977951008,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":5426930320},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,3,8,11,14,16,4,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18608128,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4544,"num_read_kb":18160,"num_write":4548,"num_write_kb":18192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001929"},"pg_stats":[{"pgid":"2.7","version":"135'273385","reported_seq":544446,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:55.698966+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:23:55.698966+0000","last_peered":"2023-10-14T01:23:55.698966+0000","last_clean":"2023-10-14T01:23:55.698966+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:23:55.698966+0000","last_undegraded":"2023-10-14T01:23:55.698966+0000","last_fullsized":"2023-10-14T01:23:55.698966+0000","mapping_epoch":13,"log_start":"132'263311","ondisk_log_start":"132'263311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3336376503,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":271901,"num_read_kb":1072436,"num_write":272329,"num_write_kb":1089301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"135'287565","reported_seq":567947,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:55.721610+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:23:55.721610+0000","last_peered":"2023-10-14T01:23:55.721610+0000","last_clean":"2023-10-14T01:23:55.721610+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:23:55.721610+0000","last_undegraded":"2023-10-14T01:23:55.721610+0000","last_fullsized":"2023-10-14T01:23:55.721610+0000","mapping_epoch":13,"log_start":"132'277508","ondisk_log_start":"132'277508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3363221771,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":282438,"num_read_kb":1092037,"num_write":273346,"num_write_kb":1092983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"135'294281","reported_seq":584052,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:55.707330+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:23:55.707330+0000","last_peered":"2023-10-14T01:23:55.707330+0000","last_clean":"2023-10-14T01:23:55.707330+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:23:55.707330+0000","last_undegraded":"2023-10-14T01:23:55.707330+0000","last_fullsized":"2023-10-14T01:23:55.707330+0000","mapping_epoch":13,"log_start":"132'284213","ondisk_log_start":"132'284213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1271,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 476s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3535917611,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":290422,"num_read_kb":1141778,"num_write":287891,"num_write_kb":1151443,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"135'311385","reported_seq":611889,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:55.701133+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:23:55.701133+0000","last_peered":"2023-10-14T01:23:55.701133+0000","last_clean":"2023-10-14T01:23:55.701133+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:23:55.701133+0000","last_undegraded":"2023-10-14T01:23:55.701133+0000","last_fullsized":"2023-10-14T01:23:55.701133+0000","mapping_epoch":13,"log_start":"132'301313","ondisk_log_start":"132'301313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1432,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 548s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3633836463,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":301183,"num_read_kb":1171315,"num_write":296267,"num_write_kb":1184750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"135'299132","reported_seq":598340,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:54.001822+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:23:54.001822+0000","last_peered":"2023-10-14T01:23:54.001822+0000","last_clean":"2023-10-14T01:23:54.001822+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:23:54.001822+0000","last_undegraded":"2023-10-14T01:23:54.001822+0000","last_fullsized":"2023-10-14T01:23:54.001822+0000","mapping_epoch":13,"log_start":"132'289112","ondisk_log_start":"132'289112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3579228631,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":299298,"num_read_kb":1156719,"num_write":291719,"num_write_kb":1166369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"135'314570","reported_seq":625728,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:57.812315+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:23:57.812315+0000","last_peered":"2023-10-14T01:23:57.812315+0000","last_clean":"2023-10-14T01:23:57.812315+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:23:57.812315+0000","last_undegraded":"2023-10-14T01:23:57.812315+0000","last_fullsized":"2023-10-14T01:23:57.812315+0000","mapping_epoch":13,"log_start":"132'304512","ondisk_log_start":"132'304512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3800437156,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":311608,"num_read_kb":1220568,"num_write":308114,"num_write_kb":1232282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"135'311431","reported_seq":618007,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:57.799993+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:23:57.799993+0000","last_peered":"2023-10-14T01:23:57.799993+0000","last_clean":"2023-10-14T01:23:57.799993+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:23:57.799993+0000","last_undegraded":"2023-10-14T01:23:57.799993+0000","last_fullsized":"2023-10-14T01:23:57.799993+0000","mapping_epoch":13,"log_start":"132'301415","ondisk_log_start":"132'301415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3749343585,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":307080,"num_read_kb":1203202,"num_write":304973,"num_write_kb":1219702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"135'316984","reported_seq":620950,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:23:55.697181+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:23:55.697181+0000","last_peered":"2023-10-14T01:23:55.697181+0000","last_clean":"2023-10-14T01:23:55.697181+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:23:55.697181+0000","last_undegraded":"2023-10-14T01:23:55.697181+0000","last_fullsized":"2023-10-14T01:23:55.697181+0000","mapping_epoch":13,"log_start":"132'306916","ondisk_log_start":"132'306916","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1144,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 494s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3661681076,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":304482,"num_read_kb":1183887,"num_write":297763,"num_write_kb":1190695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":346,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:08.722459+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:23:08.722459+0000","last_peered":"2023-10-14T01:23:08.722459+0000","last_clean":"2023-10-14T01:23:08.722459+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:23:08.722459+0000","last_undegraded":"2023-10-14T01:23:08.722459+0000","last_fullsized":"2023-10-14T01:23:08.722459+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28660042796,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2368412,"num_read_kb":9241942,"num_write":2332402,"num_write_kb":9327525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19101237248,"data_stored":28977322072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80433,"ondisk_log_size":80433,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739376,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5516760,"kb_used_data":3643920,"kb_used_omap":484,"kb_used_meta":1872347,"kb_avail":88855080,"statfs":{"total":96636764160,"available":90987601920,"internally_reserved":0,"allocated":3731374080,"data_stored":5659700113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1917283652},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,2,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.53700000000000003,"15min":0.65400000000000003},"min":{"1min":0.23300000000000001,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":2.1379999999999999,"5min":4.367,"15min":6.1120000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.2430000000000001,"15min":1.0149999999999999},"min":{"1min":0.224,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.3559999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.54000000000000004}]},{"osd":1,"last update":"Sat Oct 14 01:23:45 2023","interfaces":[{"interface":"back","average":{"1min":1.998,"5min":1.847,"15min":2.089},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":9.3539999999999992,"5min":10.878,"15min":44.308},"last":0.41099999999999998},{"interface":"front","average":{"1min":2.0569999999999999,"5min":1.899,"15min":2.1899999999999999},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.27600000000000002},"max":{"1min":9.3919999999999995,"5min":10.912000000000001,"15min":44.350000000000001},"last":0.66700000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739366,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11141932,"kb_used_data":9326692,"kb_used_omap":1204,"kb_used_meta":1814027,"kb_avail":83229908,"statfs":{"total":96636764160,"available":85227425792,"internally_reserved":0,"allocated":9550532608,"data_stored":14488429860,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":1857564397},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,5,8,6,14,4,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.53200000000000003,"15min":0.76000000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":6.4020000000000001,"15min":15.132999999999999},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.0209999999999999,"15min":0.90100000000000002},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.215},"max":{"1min":3.6219999999999999,"5min":8.4670000000000005,"15min":15.173},"last":0.375}]},{"osd":2,"last update":"Sat Oct 14 01:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.165,"15min":1.006},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":2.2839999999999998,"5min":9.6020000000000003,"15min":9.6020000000000003},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.86299999999999999,"15min":0.84399999999999997},"min":{"1min":0.25700000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.774,"5min":8.202,"15min":8.202},"last":0.36299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739377,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7298588,"kb_used_data":5684500,"kb_used_omap":718,"kb_used_meta":1613361,"kb_avail":87073252,"statfs":{"total":96636764160,"available":89163010048,"internally_reserved":0,"allocated":5820928000,"data_stored":8829821035,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1652082271},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:23:14 2023","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.4650000000000001,"15min":0.996},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":10.897,"5min":10.897,"15min":19.274000000000001},"last":0.26100000000000001},{"interface":"front","average":{"1min":1.383,"5min":0.86199999999999999,"15min":0.82999999999999996},"min":{"1min":0.22,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":10.669,"5min":10.669,"15min":19.376000000000001},"last":0.23999999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.68400000000000005,"15min":0.90100000000000002},"min":{"1min":0.23200000000000001,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":7.0739999999999998,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":5.431},{"interface":"front","average":{"1min":1.012,"5min":0.78100000000000003,"15min":0.77000000000000002},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.39200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5819871232,"data_stored":8828909427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9550807040,"data_stored":14488849452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3730558976,"data_stored":5659563193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:23:59.027 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:59.028 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:59.028 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:59.029 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:59.029 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:23:59.029 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:24:00.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.309+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:00.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:00.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:00.353 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.349+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:00.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.357+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:00.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.357+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:00.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.357+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:00.361 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.357+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:00.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:00.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:00.802 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.797+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:00.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.841+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:00.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:00.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:00.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:00.893+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:01.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:01.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:01.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.065+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:01.477 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.473+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:01.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:01.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:01.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.549+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:01.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:01.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:02.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:02.705+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:02.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:02.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:02.727 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:02.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:02.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:02.733+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:02.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:02.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:04.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:04.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:04.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:04.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.225+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:04.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.225+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:04.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.233+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:04.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.757+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:04.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:04.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:04.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:04.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.845+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:04.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.845+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:04.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:04.857+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:06.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.121+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:06.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:06.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:06.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:06.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:06.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.161+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:06.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.161+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:06.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:06.161+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:07.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.449+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:07.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:07.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:07.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:07.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:07.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:07.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:07.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:07.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:07.509+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:08.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:08.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:08.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:08.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:08.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:08.989+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:09.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:09.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.133+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:09.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.905+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:09.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.905+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:09.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:09.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.949+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:09.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:09.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:09.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:09.973 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:09.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:09.969+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:10.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:10.613+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:10.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:10.637+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:12.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:12.165+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:12.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:12.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:12.195 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:12.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:12.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:12.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:12.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:12.185+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:12.218 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:12.213+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:13.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:13.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:13.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:13.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:13.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:13.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:13.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:13.125+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:13.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:13.137+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:14.047 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:14.041+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:14.052 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:14.049+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:15.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.441+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:15.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:15.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:15.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:15.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:15.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.705+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:15.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.753+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:15.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:15.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:15.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:15.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:15.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.789+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:15.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.821+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:15.831 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.825+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:15.832 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.829+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:15.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.897+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:15.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:15.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.901+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:15.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:15.901+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:17.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.773+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:17.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.785+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:17.814 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:17.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:17.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:17.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.825+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:17.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.825+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:17.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.825+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:17.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:17.825+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:18.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.745+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:18.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.765+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:18.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:18.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:18.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.837+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:18.840 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.837+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:18.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.841+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:18.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:18.845+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:19.030 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:24:19.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:19.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:19.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:19.541+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:19.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:19.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:19.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:19.609+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:20.016 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:24:20.017 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:24:20.048 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2674,"stamp":"2023-10-14T01:24:18.884669+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28693843532,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2376668,"num_read_kb":9274447,"num_write":2340703,"num_write_kb":9362601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80619,"ondisk_log_size":80619,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24284088,"kb_used_data":18720924,"kb_used_omap":2407,"kb_used_meta":5560728,"kb_avail":258831432,"statfs":{"total":289910292480,"available":265043386368,"internally_reserved":0,"allocated":19170226176,"data_stored":29045342496,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2465136,"internal_metadata":5694186128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,3,11,14,20,8,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18673664,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4686,"num_read_kb":18664,"num_write":4570,"num_write_kb":18280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005977"},"pg_stats":[{"pgid":"2.7","version":"135'274289","reported_seq":546255,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:24:16.241784+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:24:16.241784+0000","last_peered":"2023-10-14T01:24:16.241784+0000","last_clean":"2023-10-14T01:24:16.241784+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:24:16.241784+0000","last_undegraded":"2023-10-14T01:24:16.241784+0000","last_fullsized":"2023-10-14T01:24:16.241784+0000","mapping_epoch":13,"log_start":"132'264211","ondisk_log_start":"132'264211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3340071095,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":272806,"num_read_kb":1076036,"num_write":273233,"num_write_kb":1092917,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"135'288587","reported_seq":569942,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:24:16.263149+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:24:16.263149+0000","last_peered":"2023-10-14T01:24:16.263149+0000","last_clean":"2023-10-14T01:24:16.263149+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:24:16.263149+0000","last_undegraded":"2023-10-14T01:24:16.263149+0000","last_fullsized":"2023-10-14T01:24:16.263149+0000","mapping_epoch":13,"log_start":"132'278508","ondisk_log_start":"132'278508","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3367145739,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":283398,"num_read_kb":1095865,"num_write":274308,"num_write_kb":1096831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"135'295373","reported_seq":586199,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:24:16.264377+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:24:16.264377+0000","last_peered":"2023-10-14T01:24:16.264377+0000","last_clean":"2023-10-14T01:24:16.264377+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:24:16.264377+0000","last_undegraded":"2023-10-14T01:24:16.264377+0000","last_fullsized":"2023-10-14T01:24:16.264377+0000","mapping_epoch":13,"log_start":"133'285313","ondisk_log_start":"133'285313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1323,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 496s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3540169259,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":291466,"num_read_kb":1145938,"num_write":288929,"num_write_kb":1155595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"135'312506","reported_seq":614064,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:24:16.184775+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:24:16.184775+0000","last_peered":"2023-10-14T01:24:16.184775+0000","last_clean":"2023-10-14T01:24:16.184775+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:24:16.184775+0000","last_undegraded":"2023-10-14T01:24:16.184775+0000","last_fullsized":"2023-10-14T01:24:16.184775+0000","mapping_epoch":13,"log_start":"133'302413","ondisk_log_start":"133'302413","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1469,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 568s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3638092207,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":302217,"num_read_kb":1175435,"num_write":297310,"num_write_kb":1188922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"135'300178","reported_seq":600435,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:24:14.248107+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:24:14.248107+0000","last_peered":"2023-10-14T01:24:14.248107+0000","last_clean":"2023-10-14T01:24:14.248107+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:24:14.248107+0000","last_undegraded":"2023-10-14T01:24:14.248107+0000","last_fullsized":"2023-10-14T01:24:14.248107+0000","mapping_epoch":13,"log_start":"132'290112","ondisk_log_start":"132'290112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3583422935,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":300326,"num_read_kb":1160811,"num_write":292745,"num_write_kb":1170473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"135'315614","reported_seq":627866,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:24:17.991449+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:24:17.991449+0000","last_peered":"2023-10-14T01:24:17.991449+0000","last_clean":"2023-10-14T01:24:17.991449+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:24:17.991449+0000","last_undegraded":"2023-10-14T01:24:17.991449+0000","last_fullsized":"2023-10-14T01:24:17.991449+0000","mapping_epoch":13,"log_start":"133'305612","ondisk_log_start":"133'305612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3804672420,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":312688,"num_read_kb":1224864,"num_write":309150,"num_write_kb":1236426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"135'312511","reported_seq":620136,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:24:17.984353+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:24:17.984353+0000","last_peered":"2023-10-14T01:24:17.984353+0000","last_clean":"2023-10-14T01:24:17.984353+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:24:17.984353+0000","last_undegraded":"2023-10-14T01:24:17.984353+0000","last_fullsized":"2023-10-14T01:24:17.984353+0000","mapping_epoch":13,"log_start":"133'302415","ondisk_log_start":"133'302415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3753693537,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":308119,"num_read_kb":1207354,"num_write":306045,"num_write_kb":1223990,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"135'318135","reported_seq":623168,"reported_epoch":135,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:24:16.179124+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:24:16.179124+0000","last_peered":"2023-10-14T01:24:16.179124+0000","last_clean":"2023-10-14T01:24:16.179124+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:24:16.179124+0000","last_undegraded":"2023-10-14T01:24:16.179124+0000","last_fullsized":"2023-10-14T01:24:16.179124+0000","mapping_epoch":13,"log_start":"133'308117","ondisk_log_start":"133'308117","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1184,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 515s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3665985972,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":305522,"num_read_kb":1188035,"num_write":298814,"num_write_kb":1194899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":346,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:08.722459+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:23:08.722459+0000","last_peered":"2023-10-14T01:23:08.722459+0000","last_clean":"2023-10-14T01:23:08.722459+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:23:08.722459+0000","last_undegraded":"2023-10-14T01:23:08.722459+0000","last_fullsized":"2023-10-14T01:23:08.722459+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28693253164,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2376542,"num_read_kb":9274338,"num_write":2340534,"num_write_kb":9360053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19167817728,"data_stored":29043902552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80492,"ondisk_log_size":80492,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739380,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5543072,"kb_used_data":3656344,"kb_used_omap":484,"kb_used_meta":1886235,"kb_avail":88828768,"statfs":{"total":96636764160,"available":90960658432,"internally_reserved":0,"allocated":3744096256,"data_stored":5672422289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1931504964},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,1,6,6,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.53700000000000003,"15min":0.65400000000000003},"min":{"1min":0.23300000000000001,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":2.1379999999999999,"5min":4.367,"15min":6.1120000000000001},"last":0.27400000000000002},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.2430000000000001,"15min":1.0149999999999999},"min":{"1min":0.224,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.3559999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.16400000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:23:45 2023","interfaces":[{"interface":"back","average":{"1min":1.998,"5min":1.847,"15min":2.089},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":9.3539999999999992,"5min":10.878,"15min":44.308},"last":0.85199999999999998},{"interface":"front","average":{"1min":2.0569999999999999,"5min":1.899,"15min":2.1899999999999999},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.27600000000000002},"max":{"1min":9.3919999999999995,"5min":10.912000000000001,"15min":44.350000000000001},"last":0.89100000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739370,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11346540,"kb_used_data":9360544,"kb_used_omap":1204,"kb_used_meta":1984779,"kb_avail":83025300,"statfs":{"total":96636764160,"available":85017907200,"internally_reserved":0,"allocated":9585197056,"data_stored":14523094308,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1233171,"internal_metadata":2032414445},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,9,7,7,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:23:20 2023","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.53200000000000003,"15min":0.76000000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.18099999999999999},"max":{"1min":0.45600000000000002,"5min":6.4020000000000001,"15min":15.132999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":1.0209999999999999,"15min":0.90100000000000002},"min":{"1min":0.26700000000000002,"5min":0.23999999999999999,"15min":0.215},"max":{"1min":3.6219999999999999,"5min":8.4670000000000005,"15min":15.173},"last":0.44900000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:23:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":1.165,"15min":1.006},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.22500000000000001},"max":{"1min":2.2839999999999998,"5min":9.6020000000000003,"15min":9.6020000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.86299999999999999,"15min":0.84399999999999997},"min":{"1min":0.25700000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":3.774,"5min":8.202,"15min":8.202},"last":0.26600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739381,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7394476,"kb_used_data":5704036,"kb_used_omap":718,"kb_used_meta":1689713,"kb_avail":86977364,"statfs":{"total":96636764160,"available":89064820736,"internally_reserved":0,"allocated":5840932864,"data_stored":8849825899,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1730266719},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:23:14 2023","interfaces":[{"interface":"back","average":{"1min":1.9590000000000001,"5min":1.4650000000000001,"15min":0.996},"min":{"1min":0.19400000000000001,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":10.897,"5min":10.897,"15min":19.274000000000001},"last":0.23799999999999999},{"interface":"front","average":{"1min":1.383,"5min":0.86199999999999999,"15min":0.82999999999999996},"min":{"1min":0.22,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":10.669,"5min":10.669,"15min":19.376000000000001},"last":0.26400000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:23:14 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.68400000000000005,"15min":0.90100000000000002},"min":{"1min":0.23200000000000001,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":7.0739999999999998,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.28499999999999998},{"interface":"front","average":{"1min":1.012,"5min":0.78100000000000003,"15min":0.77000000000000002},"min":{"1min":0.255,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.86899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5839990784,"data_stored":8849028979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9584373760,"data_stored":14522416172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3743453184,"data_stored":5672457401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:24:20.050 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:20.050 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:20.051 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:20.051 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:20.052 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:20.052 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:24:22.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.273+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:22.313 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.309+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:22.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:22.317 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:22.330 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:22.321+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:22.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:22.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:22.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:22.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:22.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:22.345+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:22.348 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:22.345+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:23.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:23.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:23.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.433+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:23.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.433+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:23.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.433+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:23.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.457+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:23.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.789+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:23.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:23.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:23.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:23.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:23.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.833+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:23.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.833+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:23.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:23.837+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:24.608 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:24.605+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:24:24.609 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:24.605+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:24.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:24.869+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:24:24.882 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:24.877+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:24:24.904 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:24.901+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:24.909 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:24.905+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:25.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:25.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.864+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:25.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:25.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.904+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:25.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:25.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:25.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.908+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:25.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:25.908+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:26.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.588+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:24:26.595 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.588+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:26.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:26.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:26.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:26.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:26.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:26.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.628+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:26.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.628+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:26.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.628+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:26.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:26.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:26.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:26.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:26.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:26.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.712+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:26.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:26.716+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:27.755 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:27.748+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:29.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.156+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:29.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:29.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:29.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:29.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:29.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:29.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.228+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:29.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.228+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:29.236 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:29.228+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:29.660 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:29.656+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:24:29.683 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:29.680+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:30.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.000+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:30.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:30.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:30.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:30.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:30.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:30.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.072+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:30.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.072+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:30.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:30.080+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:31.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:31.480+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:31.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:31.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:31.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:31.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:31.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:31.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:31.536 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:31.532+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:32.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:32.160+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:32.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:32.164+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:32.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:32.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:32.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:32.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:32.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:32.216+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:33.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:33.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:33.821 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:33.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:33.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:33.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:33.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:33.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:33.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:33.868+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:33.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:33.868+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:33.967 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:33.960+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:33.978 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:33.972+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:34.161 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:34.152+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:34.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.288+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:34.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:34.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:34.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:34.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:34.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.360+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:34.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.360+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:34.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.360+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:34.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:34.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:34.648+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:35.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.732+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:35.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:35.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:35.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:35.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:35.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.820+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:35.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.820+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:35.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:35.820+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:37.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:37.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:37.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:37.728+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:37.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:37.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:37.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:37.740+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:37.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:37.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:37.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:37.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:37.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:37.840+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:40.053 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:24:40.874 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:24:40.875 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:24:40.991 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2685,"stamp":"2023-10-14T01:24:38.888104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28726447692,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2384544,"num_read_kb":9305835,"num_write":2348658,"num_write_kb":9394421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80652,"ondisk_log_size":80652,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24371288,"kb_used_data":18783292,"kb_used_omap":2473,"kb_used_meta":5585494,"kb_avail":258744232,"statfs":{"total":289910292480,"available":264954093568,"internally_reserved":0,"allocated":19234091008,"data_stored":29109193696,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2532518,"internal_metadata":5719546714},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,3,12,16,22,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18640896,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4581,"num_read_kb":18264,"num_write":4549,"num_write_kb":18196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002041"},"pg_stats":[{"pgid":"2.7","version":"136'275196","reported_seq":548070,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:24:36.928996+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:24:36.928996+0000","last_peered":"2023-10-14T01:24:36.928996+0000","last_clean":"2023-10-14T01:24:36.928996+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:24:36.928996+0000","last_undegraded":"2023-10-14T01:24:36.928996+0000","last_fullsized":"2023-10-14T01:24:36.928996+0000","mapping_epoch":13,"log_start":"132'265111","ondisk_log_start":"132'265111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3343773879,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":273713,"num_read_kb":1079648,"num_write":274140,"num_write_kb":1096545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"136'289618","reported_seq":571876,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:24:36.888098+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:24:36.888098+0000","last_peered":"2023-10-14T01:24:36.888098+0000","last_clean":"2023-10-14T01:24:36.888098+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:24:36.888098+0000","last_undegraded":"2023-10-14T01:24:36.888098+0000","last_fullsized":"2023-10-14T01:24:36.888098+0000","mapping_epoch":13,"log_start":"133'279608","ondisk_log_start":"133'279608","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3371122955,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":284288,"num_read_kb":1099421,"num_write":275279,"num_write_kb":1100715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"136'296389","reported_seq":588210,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:24:36.854163+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:24:36.854163+0000","last_peered":"2023-10-14T01:24:36.854163+0000","last_clean":"2023-10-14T01:24:36.854163+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:24:36.854163+0000","last_undegraded":"2023-10-14T01:24:36.854163+0000","last_fullsized":"2023-10-14T01:24:36.854163+0000","mapping_epoch":13,"log_start":"133'286313","ondisk_log_start":"133'286313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1391,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 517s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3544064555,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":292442,"num_read_kb":1149830,"num_write":289877,"num_write_kb":1159387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"136'313591","reported_seq":616202,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:24:36.899145+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:24:36.899145+0000","last_peered":"2023-10-14T01:24:36.899145+0000","last_clean":"2023-10-14T01:24:36.899145+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:24:36.899145+0000","last_undegraded":"2023-10-14T01:24:36.899145+0000","last_fullsized":"2023-10-14T01:24:36.899145+0000","mapping_epoch":13,"log_start":"133'303513","ondisk_log_start":"133'303513","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1513,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 589s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3642192303,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":303249,"num_read_kb":1179531,"num_write":298308,"num_write_kb":1192914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"136'301180","reported_seq":602462,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:24:34.406221+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:24:34.406221+0000","last_peered":"2023-10-14T01:24:34.406221+0000","last_clean":"2023-10-14T01:24:34.406221+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:24:34.406221+0000","last_undegraded":"2023-10-14T01:24:34.406221+0000","last_fullsized":"2023-10-14T01:24:34.406221+0000","mapping_epoch":13,"log_start":"133'291112","ondisk_log_start":"133'291112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3587420631,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":301329,"num_read_kb":1164811,"num_write":293722,"num_write_kb":1174381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"136'316700","reported_seq":629998,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:24:38.221375+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:24:38.221375+0000","last_peered":"2023-10-14T01:24:38.221375+0000","last_clean":"2023-10-14T01:24:38.221375+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:24:38.221375+0000","last_undegraded":"2023-10-14T01:24:38.221375+0000","last_fullsized":"2023-10-14T01:24:38.221375+0000","mapping_epoch":13,"log_start":"133'306612","ondisk_log_start":"133'306612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3809030564,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":313727,"num_read_kb":1229004,"num_write":310220,"num_write_kb":1240706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"136'313570","reported_seq":622225,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:24:38.171475+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:24:38.171475+0000","last_peered":"2023-10-14T01:24:38.171475+0000","last_clean":"2023-10-14T01:24:38.171475+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:24:38.171475+0000","last_undegraded":"2023-10-14T01:24:38.171475+0000","last_fullsized":"2023-10-14T01:24:38.171475+0000","mapping_epoch":13,"log_start":"133'303515","ondisk_log_start":"133'303515","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3757969761,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":309139,"num_read_kb":1211426,"num_write":307088,"num_write_kb":1228162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"136'319282","reported_seq":625349,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:24:36.885724+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:24:36.885724+0000","last_peered":"2023-10-14T01:24:36.885724+0000","last_clean":"2023-10-14T01:24:36.885724+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:24:36.885724+0000","last_undegraded":"2023-10-14T01:24:36.885724+0000","last_fullsized":"2023-10-14T01:24:36.885724+0000","mapping_epoch":13,"log_start":"133'309217","ondisk_log_start":"133'309217","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1227,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 536s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3670282676,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":306531,"num_read_kb":1192055,"num_write":299855,"num_write_kb":1199063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":347,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:35.173958+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:23:35.173958+0000","last_peered":"2023-10-14T01:23:35.173958+0000","last_clean":"2023-10-14T01:23:35.173958+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:23:35.173958+0000","last_undegraded":"2023-10-14T01:23:35.173958+0000","last_fullsized":"2023-10-14T01:23:35.173958+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28725857324,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2384418,"num_read_kb":9305726,"num_write":2348489,"num_write_kb":9391873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19232866304,"data_stored":29108951128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80525,"ondisk_log_size":80525,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739384,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5622092,"kb_used_data":3668740,"kb_used_omap":484,"kb_used_meta":1952859,"kb_avail":88749748,"statfs":{"total":96636764160,"available":90879741952,"internally_reserved":0,"allocated":3756789760,"data_stored":5685111249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":1999727940},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.53700000000000003,"15min":0.65400000000000003},"min":{"1min":0.23300000000000001,"5min":0.182,"15min":0.17699999999999999},"max":{"1min":2.1379999999999999,"5min":4.367,"15min":6.1120000000000001},"last":0.26200000000000001},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.2430000000000001,"15min":1.0149999999999999},"min":{"1min":0.224,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":5.3559999999999999,"5min":11.481999999999999,"15min":11.554},"last":0.307}]},{"osd":1,"last update":"Sat Oct 14 01:23:45 2023","interfaces":[{"interface":"back","average":{"1min":1.998,"5min":1.847,"15min":2.089},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20699999999999999},"max":{"1min":9.3539999999999992,"5min":10.878,"15min":44.308},"last":0.51700000000000002},{"interface":"front","average":{"1min":2.0569999999999999,"5min":1.899,"15min":2.1899999999999999},"min":{"1min":0.28100000000000003,"5min":0.27900000000000003,"15min":0.27600000000000002},"max":{"1min":9.3919999999999995,"5min":10.912000000000001,"15min":44.350000000000001},"last":0.58399999999999996}]}]},{"osd":1,"up_from":8,"seq":34359739374,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11254916,"kb_used_data":9391864,"kb_used_omap":1270,"kb_used_meta":1861769,"kb_avail":83116924,"statfs":{"total":96636764160,"available":85111730176,"internally_reserved":0,"allocated":9617268736,"data_stored":14555161444,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1300553,"internal_metadata":1906452407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,0,1,7,11,13,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57199999999999995,"15min":0.70999999999999996},"min":{"1min":0.23300000000000001,"5min":0.20100000000000001,"15min":0.18099999999999999},"max":{"1min":7.2549999999999999,"5min":7.2549999999999999,"15min":15.132999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0589999999999999,"15min":0.91800000000000004},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.33,"5min":10.33,"15min":15.173},"last":0.33500000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:24:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.077,"15min":0.98699999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.22500000000000001},"max":{"1min":10.375999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.89300000000000002,"15min":0.77400000000000002},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.308,"5min":10.308,"15min":10.308},"last":0.379}]}]},{"osd":0,"up_from":8,"seq":34359739385,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7494280,"kb_used_data":5722688,"kb_used_omap":718,"kb_used_meta":1770865,"kb_avail":86877560,"statfs":{"total":96636764160,"available":88962621440,"internally_reserved":0,"allocated":5860032512,"data_stored":8868921003,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1813366367},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.20200000000000001,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":6.4349999999999996,"5min":10.897,"15min":19.274000000000001},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.875,"5min":0.91600000000000004,"15min":0.86399999999999999},"min":{"1min":0.20999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.7999999999999998,"5min":10.669,"15min":19.376000000000001},"last":0.34000000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.871},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":5.431,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.249,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":4.8970000000000002,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5859381248,"data_stored":8868419443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9617182720,"data_stored":14555225132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3756302336,"data_stored":5685306553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:24:40.993 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:40.993 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:40.994 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:40.994 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:40.994 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:24:40.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:24:41.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:41.016+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:41.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:41.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:41.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:41.052+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:41.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:41.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:41.786 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:24:41.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:24:42.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:42.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:42.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:42.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.332+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:42.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.332+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:42.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.332+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:42.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.332+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:42.707 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:42.704+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:42.727 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:24:42.738 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:42.732+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:42.779 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.5 2023-10-14T01:24:42.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:42.940+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:43.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:43.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:43.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:43.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:43.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:43.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:43.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:43.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:43.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:43.060+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:43.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:43.060+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:43.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:43.060+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:43.807 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.5 on osd.1 to deep-scrub 2023-10-14T01:24:43.901 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.4 2023-10-14T01:24:44.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:44.652+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:44.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:44.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:44.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:44.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:44.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:44.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:44.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:44.684+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:44.821 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.4 on osd.1 to deep-scrub 2023-10-14T01:24:44.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.3 2023-10-14T01:24:45.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:45.768+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:45.789 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:45.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:45.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:45.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:45.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:45.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:45.818 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.3 on osd.1 to deep-scrub 2023-10-14T01:24:45.844 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:45.840+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:45.848 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:24:45.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:45.840+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:47.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:47.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:47.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:47.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:47.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:47.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:47.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:47.260+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:47.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:47.260+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:47.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:47.280+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:48.702 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:48.696+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:24:48.702 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:48.696+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:49.197 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.192+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:49.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:49.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.252+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:49.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:49.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:49.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.256+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:49.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.256+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:49.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:49.264+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:50.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:50.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:50.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:50.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:50.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:50.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:50.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:50.860+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:50.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:50.860+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:50.869 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:50.860+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:50.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:50.876+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:51.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:51.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:51.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:51.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:51.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:51.296+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:52.321 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:52.316+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:24:52.328 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:52.324+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:24:52.328 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:52.324+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:52.347 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:52.340+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:52.353 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:52.348+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:24:52.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:52.356+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:52.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.356+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:24:52.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.356+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:24:52.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.356+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:52.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:52.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:52.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.388+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:52.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:52.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:52.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:52.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:52.424 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:52.416+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:53.796 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:53.792+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:24:53.837 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:53.832+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:53.837 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:53.832+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:57.660 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:57.660+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:24:57.662 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:57.660+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:58.598 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:58.596+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:24:58.608 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:58.608+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:58.617 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:58.616+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:58.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.676+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:58.684 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.680+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:24:58.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.692+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:24:58.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:58.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:24:58.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:24:58.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.720+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:24:58.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.720+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:24:58.724 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:58.724+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:24:59.076 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:59.076+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:24:59.080 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:24:59.080+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:24:59.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:59.672+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:24:59.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:59.716+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:24:59.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:24:59.732+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:00.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:00.964+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:01.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:01.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:01.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:01.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:01.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:01.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:01.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:01.004+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:04.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:04.539+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:04.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:04.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:04.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:04.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:04.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:04.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:04.577 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:04.575+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:04.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:04.575+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:04.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:04.575+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:05.060 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:05.059+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:25:05.060 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:05.059+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:25:05.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:05.237 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.235+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:05.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:05.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:05.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:05.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.507+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:05.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.507+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:05.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.511+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:05.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.511+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:05.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.519+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:05.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:05.538 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:05.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.543+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:05.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.719+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:05.739 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:05.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:05.741 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.739+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:05.742 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.739+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:05.788 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.787+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:05.849 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:25:05.910 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.907+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:05.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:05.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:05.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.947+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:05.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.947+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:05.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:05.959+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:06.499 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:25:06.500 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:25:06.547 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2698,"stamp":"2023-10-14T01:25:04.895755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28768636492,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2394575,"num_read_kb":9345879,"num_write":2358961,"num_write_kb":9435633,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80414,"ondisk_log_size":80414,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24857628,"kb_used_data":18869508,"kb_used_omap":2473,"kb_used_meta":5985622,"kb_avail":258257892,"statfs":{"total":289910292480,"available":264456081408,"internally_reserved":0,"allocated":19322376192,"data_stored":29197478880,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2532518,"internal_metadata":6129277786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,9,8,13,13,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20045824,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4839,"num_read_kb":19328,"num_write":4894,"num_write_kb":19576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002395"},"pg_stats":[{"pgid":"2.7","version":"136'276337","reported_seq":550348,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:02.633071+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:25:02.633071+0000","last_peered":"2023-10-14T01:25:02.633071+0000","last_clean":"2023-10-14T01:25:02.633071+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:25:02.633071+0000","last_undegraded":"2023-10-14T01:25:02.633071+0000","last_fullsized":"2023-10-14T01:25:02.633071+0000","mapping_epoch":13,"log_start":"133'266311","ondisk_log_start":"133'266311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3348471991,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":274850,"num_read_kb":1084192,"num_write":275281,"num_write_kb":1101109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"136'290856","reported_seq":574271,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:02.620598+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:25:02.620598+0000","last_peered":"2023-10-14T01:25:02.620598+0000","last_clean":"2023-10-14T01:25:02.620598+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:25:02.620598+0000","last_undegraded":"2023-10-14T01:25:02.620598+0000","last_fullsized":"2023-10-14T01:25:02.620598+0000","mapping_epoch":13,"log_start":"133'280808","ondisk_log_start":"133'280808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3376009483,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":285428,"num_read_kb":1103981,"num_write":276471,"num_write_kb":1105483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"136'297638","reported_seq":590689,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:02.606285+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:25:02.606285+0000","last_peered":"2023-10-14T01:25:02.606285+0000","last_clean":"2023-10-14T01:25:02.606285+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:25:02.606285+0000","last_undegraded":"2023-10-14T01:25:02.606285+0000","last_fullsized":"2023-10-14T01:25:02.606285+0000","mapping_epoch":13,"log_start":"133'287614","ondisk_log_start":"133'287614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1474,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 542s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3548869163,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":293660,"num_read_kb":1154694,"num_write":291053,"num_write_kb":1164091,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"136'314986","reported_seq":618888,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:02.550943+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:25:02.550943+0000","last_peered":"2023-10-14T01:25:02.550943+0000","last_clean":"2023-10-14T01:25:02.550943+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:25:02.550943+0000","last_undegraded":"2023-10-14T01:25:02.550943+0000","last_fullsized":"2023-10-14T01:25:02.550943+0000","mapping_epoch":13,"log_start":"133'304913","ondisk_log_start":"133'304913","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1577,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 615s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3647545775,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":304516,"num_read_kb":1184583,"num_write":299618,"num_write_kb":1198154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"136'302737","reported_seq":605417,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:04.690615+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:25:04.690615+0000","last_peered":"2023-10-14T01:25:04.690615+0000","last_clean":"2023-10-14T01:25:04.690615+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:25:04.690615+0000","last_undegraded":"2023-10-14T01:25:04.690615+0000","last_fullsized":"2023-10-14T01:25:04.690615+0000","mapping_epoch":13,"log_start":"133'292712","ondisk_log_start":"133'292712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3593638359,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":302697,"num_read_kb":1170267,"num_write":295244,"num_write_kb":1180469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"136'318019","reported_seq":632619,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:03.553480+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:25:03.553480+0000","last_peered":"2023-10-14T01:25:03.553480+0000","last_clean":"2023-10-14T01:25:03.553480+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:25:03.553480+0000","last_undegraded":"2023-10-14T01:25:03.553480+0000","last_fullsized":"2023-10-14T01:25:03.553480+0000","mapping_epoch":13,"log_start":"133'308012","ondisk_log_start":"133'308012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3814355364,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":315019,"num_read_kb":1234160,"num_write":311521,"num_write_kb":1245910,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"136'314921","reported_seq":624898,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:03.513169+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:25:03.513169+0000","last_peered":"2023-10-14T01:25:03.513169+0000","last_clean":"2023-10-14T01:25:03.513169+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:25:03.513169+0000","last_undegraded":"2023-10-14T01:25:03.513169+0000","last_fullsized":"2023-10-14T01:25:03.513169+0000","mapping_epoch":13,"log_start":"133'304915","ondisk_log_start":"133'304915","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3763425633,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":310452,"num_read_kb":1216670,"num_write":308421,"num_write_kb":1233494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"136'320695","reported_seq":628086,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:02.543418+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:25:02.543418+0000","last_peered":"2023-10-14T01:25:02.543418+0000","last_clean":"2023-10-14T01:25:02.543418+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:25:02.543418+0000","last_undegraded":"2023-10-14T01:25:02.543418+0000","last_fullsized":"2023-10-14T01:25:02.543418+0000","mapping_epoch":13,"log_start":"133'310617","ondisk_log_start":"133'310617","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1270,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 561s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3675730356,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":307827,"num_read_kb":1197223,"num_write":301183,"num_write_kb":1204375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":347,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:35.173958+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:23:35.173958+0000","last_peered":"2023-10-14T01:23:35.173958+0000","last_clean":"2023-10-14T01:23:35.173958+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:23:35.173958+0000","last_undegraded":"2023-10-14T01:23:35.173958+0000","last_fullsized":"2023-10-14T01:23:35.173958+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28768046124,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2394449,"num_read_kb":9345770,"num_write":2358792,"num_write_kb":9433085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19319934976,"data_stored":29196019800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80287,"ondisk_log_size":80287,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739389,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5735300,"kb_used_data":3684860,"kb_used_omap":484,"kb_used_meta":2049947,"kb_avail":88636540,"statfs":{"total":96636764160,"available":90763816960,"internally_reserved":0,"allocated":3773296640,"data_stored":5701618129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":2099146052},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,5,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.48299999999999998,"15min":0.64000000000000001},"min":{"1min":0.20399999999999999,"5min":0.182,"15min":0.17799999999999999},"max":{"1min":0.77000000000000002,"5min":3.9180000000000001,"15min":6.1120000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":1.1180000000000001,"15min":0.96399999999999997},"min":{"1min":0.16400000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":3.427,"5min":11.481999999999999,"15min":11.481999999999999},"last":0.45900000000000002}]},{"osd":1,"last update":"Sat Oct 14 01:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.74,"15min":2.0350000000000001},"min":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.8780000000000001,"5min":10.878,"15min":44.308},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.8029999999999999,"15min":2.101},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":5.6980000000000004,"5min":10.912000000000001,"15min":44.350000000000001},"last":0.84099999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739379,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11495380,"kb_used_data":9433356,"kb_used_omap":1270,"kb_used_meta":2060745,"kb_avail":82876460,"statfs":{"total":96636764160,"available":84865495040,"internally_reserved":0,"allocated":9659756544,"data_stored":14597649252,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1300553,"internal_metadata":2110203831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,7,2,5,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57199999999999995,"15min":0.70999999999999996},"min":{"1min":0.23300000000000001,"5min":0.20100000000000001,"15min":0.18099999999999999},"max":{"1min":7.2549999999999999,"5min":7.2549999999999999,"15min":15.132999999999999},"last":1.73},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0589999999999999,"15min":0.91800000000000004},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.33,"5min":10.33,"15min":15.173},"last":1.77}]},{"osd":2,"last update":"Sat Oct 14 01:24:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.077,"15min":0.98699999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.22500000000000001},"max":{"1min":10.375999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":1.6859999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.89300000000000002,"15min":0.77400000000000002},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.308,"5min":10.308,"15min":10.308},"last":0.502}]}]},{"osd":0,"up_from":8,"seq":34359739391,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7626948,"kb_used_data":5751292,"kb_used_omap":718,"kb_used_meta":1874929,"kb_avail":86744892,"statfs":{"total":96636764160,"available":88826769408,"internally_reserved":0,"allocated":5889323008,"data_stored":8898211499,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":1919927903},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.20200000000000001,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":6.4349999999999996,"5min":10.897,"15min":19.274000000000001},"last":1.996},{"interface":"front","average":{"1min":0.875,"5min":0.91600000000000004,"15min":0.86399999999999999},"min":{"1min":0.20999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.7999999999999998,"5min":10.669,"15min":19.376000000000001},"last":0.375}]},{"osd":2,"last update":"Sat Oct 14 01:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.871},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":5.431,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.249,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":4.8970000000000002,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.41199999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5888892928,"data_stored":8897931123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9658667008,"data_stored":14596709420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3772375040,"data_stored":5701379257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:25:06.548 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:06.549 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:06.549 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:06.549 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:06.550 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:06.550 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:25:06.953 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:06.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:06.981 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:06.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:06.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:06.979+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:06.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:06.983+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:07.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:07.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:07.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:07.151+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:08.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:08.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:08.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:08.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:08.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:08.339+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:08.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:08.339+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:08.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:08.339+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:08.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:08.339+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:09.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:09.107+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:09.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:09.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:09.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:09.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:09.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:09.175+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:10.127 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.123+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:10.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:10.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:10.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:10.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:10.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:10.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.167+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:10.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.167+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:10.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.167+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:10.591 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:10.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.611+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:10.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:10.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:10.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.651+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:10.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.655+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:10.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:10.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:10.911 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:10.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.931+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:10.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:10.931+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:11.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:11.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:11.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:11.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:11.847 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:11.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:11.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:11.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:11.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:11.843+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:12.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:12.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:12.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.091+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:12.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.103+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:12.444 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:12.443+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:25:12.477 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:12.475+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:25:12.500 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:12.499+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:25:12.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.895+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:25:12.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.895+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:25:12.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.895+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:12.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:12.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:12.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:12.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:12.915+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:13.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:13.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:13.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.379+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:13.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.403+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:13.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.495+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:13.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:13.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:13.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:13.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.547+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:13.551 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.547+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:13.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.563+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:13.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:13.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:13.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:13.571+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:15.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:15.455+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:15.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:15.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:15.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:15.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:15.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:15.507+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:15.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:15.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:15.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:15.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:15.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:15.651+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:18.997 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:18.995+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:19.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:19.084 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:19.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.083+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:19.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:19.100 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.095+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:19.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.095+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:19.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.095+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:19.101 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.095+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:19.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:19.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:19.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:19.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:19.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.771+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:19.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.771+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:19.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:19.771+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:21.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:21.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:22.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:21.999+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:22.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:22.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:22.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:22.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:22.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:22.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:22.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:22.071+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:23.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:23.343+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:23.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:23.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:23.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:23.351+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:23.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:23.371+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:26.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:26.103+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:26.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:26.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:26.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:26.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:26.168 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:26.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:26.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:26.167+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:26.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:26.167+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:26.169 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:26.167+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:26.551 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:25:27.258 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:25:27.259 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:25:27.369 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2709,"stamp":"2023-10-14T01:25:26.902031+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28801117772,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2402184,"num_read_kb":9376271,"num_write":2366879,"num_write_kb":9467305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80528,"ondisk_log_size":80528,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25033984,"kb_used_data":18932136,"kb_used_omap":2473,"kb_used_meta":6099350,"kb_avail":258081536,"statfs":{"total":289910292480,"available":264275492864,"internally_reserved":0,"allocated":19386507264,"data_stored":29261609952,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2532518,"internal_metadata":6245735258},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,5,5,6,7,15,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16211968,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3964,"num_read_kb":15840,"num_write":3953,"num_write_kb":15812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004728"},"pg_stats":[{"pgid":"2.7","version":"136'277247","reported_seq":552112,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:23.062033+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:25:23.062033+0000","last_peered":"2023-10-14T01:25:23.062033+0000","last_clean":"2023-10-14T01:25:23.062033+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:25:23.062033+0000","last_undegraded":"2023-10-14T01:25:23.062033+0000","last_fullsized":"2023-10-14T01:25:23.062033+0000","mapping_epoch":13,"log_start":"133'267211","ondisk_log_start":"133'267211","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3352191159,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":275706,"num_read_kb":1087616,"num_write":276191,"num_write_kb":1104749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"136'291875","reported_seq":576229,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:23.058454+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:25:23.058454+0000","last_peered":"2023-10-14T01:25:23.058454+0000","last_clean":"2023-10-14T01:25:23.058454+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:25:23.058454+0000","last_undegraded":"2023-10-14T01:25:23.058454+0000","last_fullsized":"2023-10-14T01:25:23.058454+0000","mapping_epoch":13,"log_start":"133'281808","ondisk_log_start":"133'281808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3379867915,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":286356,"num_read_kb":1107693,"num_write":277415,"num_write_kb":1109259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"136'298648","reported_seq":592649,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:23.017981+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:25:23.017981+0000","last_peered":"2023-10-14T01:25:23.017981+0000","last_clean":"2023-10-14T01:25:23.017981+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:25:23.017981+0000","last_undegraded":"2023-10-14T01:25:23.017981+0000","last_fullsized":"2023-10-14T01:25:23.017981+0000","mapping_epoch":13,"log_start":"134'288614","ondisk_log_start":"134'288614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1559,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 563s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3552670251,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":294595,"num_read_kb":1158430,"num_write":291981,"num_write_kb":1167803,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"136'316106","reported_seq":620988,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:23.011958+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:25:23.011958+0000","last_peered":"2023-10-14T01:25:23.011958+0000","last_clean":"2023-10-14T01:25:23.011958+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:25:23.011958+0000","last_undegraded":"2023-10-14T01:25:23.011958+0000","last_fullsized":"2023-10-14T01:25:23.011958+0000","mapping_epoch":13,"log_start":"134'306013","ondisk_log_start":"134'306013","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1616,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 635s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3651756463,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":305477,"num_read_kb":1188423,"num_write":300643,"num_write_kb":1202254,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"136'303766","reported_seq":607472,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:24.847610+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:25:24.847610+0000","last_peered":"2023-10-14T01:25:24.847610+0000","last_clean":"2023-10-14T01:25:24.847610+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:25:24.847610+0000","last_undegraded":"2023-10-14T01:25:24.847610+0000","last_fullsized":"2023-10-14T01:25:24.847610+0000","mapping_epoch":13,"log_start":"134'293712","ondisk_log_start":"134'293712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3597844951,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":303703,"num_read_kb":1174283,"num_write":296263,"num_write_kb":1184545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"136'319067","reported_seq":634674,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:23.762628+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:25:23.762628+0000","last_peered":"2023-10-14T01:25:23.762628+0000","last_clean":"2023-10-14T01:25:23.762628+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:25:23.762628+0000","last_undegraded":"2023-10-14T01:25:23.762628+0000","last_fullsized":"2023-10-14T01:25:23.762628+0000","mapping_epoch":13,"log_start":"134'309012","ondisk_log_start":"134'309012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3818627492,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":316019,"num_read_kb":1238152,"num_write":312559,"num_write_kb":1250062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"136'315973","reported_seq":626914,"reported_epoch":136,"state":"active+clean","last_fresh":"2023-10-14T01:25:23.726685+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:25:23.726685+0000","last_peered":"2023-10-14T01:25:23.726685+0000","last_clean":"2023-10-14T01:25:23.726685+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:25:23.726685+0000","last_undegraded":"2023-10-14T01:25:23.726685+0000","last_fullsized":"2023-10-14T01:25:23.726685+0000","mapping_epoch":13,"log_start":"134'305915","ondisk_log_start":"134'305915","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3767697761,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":311408,"num_read_kb":1220486,"num_write":309463,"num_write_kb":1237662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"136'321821","reported_seq":630204,"reported_epoch":136,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:23.032988+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:25:23.032988+0000","last_peered":"2023-10-14T01:25:23.032988+0000","last_clean":"2023-10-14T01:25:23.032988+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:25:23.032988+0000","last_undegraded":"2023-10-14T01:25:23.032988+0000","last_fullsized":"2023-10-14T01:25:23.032988+0000","mapping_epoch":13,"log_start":"134'311817","ondisk_log_start":"134'311817","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1304,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 582s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3679871412,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":308794,"num_read_kb":1201079,"num_write":302195,"num_write_kb":1208423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":347,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-14T01:23:35.173958+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:23:35.173958+0000","last_peered":"2023-10-14T01:23:35.173958+0000","last_clean":"2023-10-14T01:23:35.173958+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:23:35.173958+0000","last_undegraded":"2023-10-14T01:23:35.173958+0000","last_fullsized":"2023-10-14T01:23:35.173958+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28800527404,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2402058,"num_read_kb":9376162,"num_write":2366710,"num_write_kb":9464757,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19384643584,"data_stored":29260728408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80401,"ondisk_log_size":80401,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739393,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5779492,"kb_used_data":3697116,"kb_used_omap":484,"kb_used_meta":2081883,"kb_avail":88592348,"statfs":{"total":96636764160,"available":90718564352,"internally_reserved":0,"allocated":3785846784,"data_stored":5714168273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":496316,"internal_metadata":2131848516},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.48299999999999998,"15min":0.64000000000000001},"min":{"1min":0.20399999999999999,"5min":0.182,"15min":0.17799999999999999},"max":{"1min":0.77000000000000002,"5min":3.9180000000000001,"15min":6.1120000000000001},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":1.1180000000000001,"15min":0.96399999999999997},"min":{"1min":0.16400000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":3.427,"5min":11.481999999999999,"15min":11.481999999999999},"last":0.252}]},{"osd":1,"last update":"Sat Oct 14 01:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.74,"15min":2.0350000000000001},"min":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.8780000000000001,"5min":10.878,"15min":44.308},"last":4.5739999999999998},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.8029999999999999,"15min":2.101},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":5.6980000000000004,"5min":10.912000000000001,"15min":44.350000000000001},"last":4.6159999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739383,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11518240,"kb_used_data":9464600,"kb_used_omap":1270,"kb_used_meta":2052361,"kb_avail":82853600,"statfs":{"total":96636764160,"available":84842086400,"internally_reserved":0,"allocated":9691750400,"data_stored":14629643108,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1300553,"internal_metadata":2101618615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,3,5,2,12,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:24:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.57199999999999995,"15min":0.70999999999999996},"min":{"1min":0.23300000000000001,"5min":0.20100000000000001,"15min":0.18099999999999999},"max":{"1min":7.2549999999999999,"5min":7.2549999999999999,"15min":15.132999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.0589999999999999,"15min":0.91800000000000004},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":10.33,"5min":10.33,"15min":15.173},"last":0.24299999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:24:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.077,"15min":0.98699999999999999},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.22500000000000001},"max":{"1min":10.375999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.89300000000000002,"15min":0.77400000000000002},"min":{"1min":0.221,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":10.308,"5min":10.308,"15min":10.308},"last":0.32200000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739395,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7736252,"kb_used_data":5770420,"kb_used_omap":718,"kb_used_meta":1965105,"kb_avail":86635588,"statfs":{"total":96636764160,"available":88714842112,"internally_reserved":0,"allocated":5908910080,"data_stored":8917798571,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":2012268127},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":1.331,"15min":1.0329999999999999},"min":{"1min":0.20200000000000001,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":6.4349999999999996,"5min":10.897,"15min":19.274000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.875,"5min":0.91600000000000004,"15min":0.86399999999999999},"min":{"1min":0.20999999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":5.7999999999999998,"5min":10.669,"15min":19.376000000000001},"last":2.343}]},{"osd":2,"last update":"Sat Oct 14 01:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.59299999999999997,"15min":0.871},"min":{"1min":0.24199999999999999,"5min":0.222,"15min":0.20200000000000001},"max":{"1min":5.431,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.78700000000000003,"15min":0.77600000000000002},"min":{"1min":0.249,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":4.8970000000000002,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5908385792,"data_stored":8917423987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9691250688,"data_stored":14629293100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3785007104,"data_stored":5714011321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:25:27.371 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:27.372 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:27.372 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:27.373 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:27.373 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:27.374 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:25:27.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:27.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:27.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:27.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:27.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:27.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.547+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:27.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.547+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:27.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:27.559+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:29.149 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:29.147+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:25:33.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:33.011+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:33.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:33.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:33.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:33.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:33.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:33.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:33.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:33.079+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:33.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:33.079+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:35.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:35.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.515+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:35.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:35.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:35.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.531+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:35.534 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.531+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:35.561 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.559+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:35.760 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:35.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:35.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:35.819+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:38.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.775+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:25:38.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.775+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:25:38.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.787+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:38.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:38.845 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:38.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.843+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:38.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:38.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:38.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.847+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:38.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.847+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:38.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:38.847+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:40.221 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:40.219+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:25:40.231 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:40.223+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:25:40.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:40.598+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:40.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:40.606+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:40.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:40.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:40.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:40.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:40.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:40.650+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:40.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:40.650+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:40.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:40.650+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:41.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:41.374+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:41.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:41.390+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:41.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:41.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:41.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:41.406+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:41.410 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:41.406+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:41.411 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:41.406+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:41.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:41.410+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:43.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:43.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:43.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:43.226+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:45.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.634+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:45.643 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:45.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.654+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:45.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:45.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.898+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:45.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.930+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:45.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:46.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:46.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:46.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.994+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:46.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.994+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:46.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:45.994+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:47.375 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:25:47.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:47.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:47.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:47.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:47.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:47.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:47.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:47.626+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:47.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:47.626+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:47.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:47.626+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:48.348 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:25:48.349 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:25:48.391 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2720,"stamp":"2023-10-14T01:25:46.908896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28834782796,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2410366,"num_read_kb":9408939,"num_write":2375107,"num_write_kb":9500217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80472,"ondisk_log_size":80472,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23394128,"kb_used_data":18998448,"kb_used_omap":2476,"kb_used_meta":4393171,"kb_avail":259721392,"statfs":{"total":289910292480,"available":265954705408,"internally_reserved":0,"allocated":19454410752,"data_stored":29329499679,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2535877,"internal_metadata":4498607675},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,8,12,17,15,9],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17661952,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4319,"num_read_kb":17252,"num_write":4314,"num_write_kb":17256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005538"},"pg_stats":[{"pgid":"2.7","version":"137'278173","reported_seq":554004,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:43.856889+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:25:43.856889+0000","last_peered":"2023-10-14T01:25:43.856889+0000","last_clean":"2023-10-14T01:25:43.856889+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:25:43.856889+0000","last_undegraded":"2023-10-14T01:25:43.856889+0000","last_fullsized":"2023-10-14T01:25:43.856889+0000","mapping_epoch":13,"log_start":"134'268111","ondisk_log_start":"134'268111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3355967671,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":276667,"num_read_kb":1091448,"num_write":277117,"num_write_kb":1108453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"137'292925","reported_seq":578238,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:43.873316+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:25:43.873316+0000","last_peered":"2023-10-14T01:25:43.873316+0000","last_clean":"2023-10-14T01:25:43.873316+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:25:43.873316+0000","last_undegraded":"2023-10-14T01:25:43.873316+0000","last_fullsized":"2023-10-14T01:25:43.873316+0000","mapping_epoch":13,"log_start":"134'282908","ondisk_log_start":"134'282908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3384017163,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":287304,"num_read_kb":1111477,"num_write":278435,"num_write_kb":1113339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"137'299752","reported_seq":594764,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:43.856353+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:25:43.856353+0000","last_peered":"2023-10-14T01:25:43.856353+0000","last_clean":"2023-10-14T01:25:43.856353+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:25:43.856353+0000","last_undegraded":"2023-10-14T01:25:43.856353+0000","last_fullsized":"2023-10-14T01:25:43.856353+0000","mapping_epoch":13,"log_start":"134'289714","ondisk_log_start":"134'289714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1608,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 584s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3557028395,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":295595,"num_read_kb":1162418,"num_write":293041,"num_write_kb":1172043,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"137'317184","reported_seq":623137,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:43.849458+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:25:43.849458+0000","last_peered":"2023-10-14T01:25:43.849458+0000","last_clean":"2023-10-14T01:25:43.849458+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:25:43.849458+0000","last_undegraded":"2023-10-14T01:25:43.849458+0000","last_fullsized":"2023-10-14T01:25:43.849458+0000","mapping_epoch":13,"log_start":"134'307113","ondisk_log_start":"134'307113","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1655,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 656s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3655942575,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":306526,"num_read_kb":1192615,"num_write":301664,"num_write_kb":1206338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"137'304761","reported_seq":609497,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:45.065042+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:25:45.065042+0000","last_peered":"2023-10-14T01:25:45.065042+0000","last_clean":"2023-10-14T01:25:45.065042+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:25:45.065042+0000","last_undegraded":"2023-10-14T01:25:45.065042+0000","last_fullsized":"2023-10-14T01:25:45.065042+0000","mapping_epoch":13,"log_start":"134'294712","ondisk_log_start":"134'294712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3601854935,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":304711,"num_read_kb":1178311,"num_write":297243,"num_write_kb":1188465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"137'320175","reported_seq":636863,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:43.961110+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:25:43.961110+0000","last_peered":"2023-10-14T01:25:43.961110+0000","last_clean":"2023-10-14T01:25:43.961110+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:25:43.961110+0000","last_undegraded":"2023-10-14T01:25:43.961110+0000","last_fullsized":"2023-10-14T01:25:43.961110+0000","mapping_epoch":13,"log_start":"134'310112","ondisk_log_start":"134'310112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3823141284,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":317090,"num_read_kb":1242432,"num_write":313663,"num_write_kb":1254478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"137'317054","reported_seq":629066,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:43.963205+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:25:43.963205+0000","last_peered":"2023-10-14T01:25:43.963205+0000","last_clean":"2023-10-14T01:25:43.963205+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:25:43.963205+0000","last_undegraded":"2023-10-14T01:25:43.963205+0000","last_fullsized":"2023-10-14T01:25:43.963205+0000","mapping_epoch":13,"log_start":"134'307015","ondisk_log_start":"134'307015","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3772096865,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":312468,"num_read_kb":1224710,"num_write":310540,"num_write_kb":1241970,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"137'322923","reported_seq":632412,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:25:43.813065+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:25:43.813065+0000","last_peered":"2023-10-14T01:25:43.813065+0000","last_clean":"2023-10-14T01:25:43.813065+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:25:43.813065+0000","last_undegraded":"2023-10-14T01:25:43.813065+0000","last_fullsized":"2023-10-14T01:25:43.813065+0000","mapping_epoch":13,"log_start":"134'312917","ondisk_log_start":"134'312917","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1336,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 603s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3684143540,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":309879,"num_read_kb":1205419,"num_write":303235,"num_write_kb":1212583,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":349,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:33.778997+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:25:33.778997+0000","last_peered":"2023-10-14T01:25:33.778997+0000","last_clean":"2023-10-14T01:25:33.778997+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:25:33.778997+0000","last_undegraded":"2023-10-14T01:25:33.778997+0000","last_fullsized":"2023-10-14T01:25:33.778997+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28834192428,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2410240,"num_read_kb":9408830,"num_write":2374938,"num_write_kb":9497669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19451904000,"data_stored":29327988824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80345,"ondisk_log_size":80345,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739397,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5156112,"kb_used_data":3709768,"kb_used_omap":486,"kb_used_meta":1445849,"kb_avail":89215728,"statfs":{"total":96636764160,"available":91356905472,"internally_reserved":0,"allocated":3798802432,"data_stored":5727119334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1480550068},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.48299999999999998,"15min":0.64000000000000001},"min":{"1min":0.20399999999999999,"5min":0.182,"15min":0.17799999999999999},"max":{"1min":0.77000000000000002,"5min":3.9180000000000001,"15min":6.1120000000000001},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":1.1180000000000001,"15min":0.96399999999999997},"min":{"1min":0.16400000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":3.427,"5min":11.481999999999999,"15min":11.481999999999999},"last":6.7039999999999997}]},{"osd":1,"last update":"Sat Oct 14 01:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.74,"15min":2.0350000000000001},"min":{"1min":0.21199999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":4.8780000000000001,"5min":10.878,"15min":44.308},"last":6.8739999999999997},{"interface":"front","average":{"1min":0.98899999999999999,"5min":1.8029999999999999,"15min":2.101},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":5.6980000000000004,"5min":10.912000000000001,"15min":44.350000000000001},"last":6.9160000000000004}]}]},{"osd":1,"up_from":8,"seq":34359739387,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10387560,"kb_used_data":9498460,"kb_used_omap":1271,"kb_used_meta":887816,"kb_avail":83984280,"statfs":{"total":96636764160,"available":85999902720,"internally_reserved":0,"allocated":9726423040,"data_stored":14664311161,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":909123880},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,6,8,6,9],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.59999999999999998,"15min":0.73599999999999999},"min":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.18099999999999999},"max":{"1min":4.4939999999999998,"5min":7.2549999999999999,"15min":15.132999999999999},"last":4.2160000000000002},{"interface":"front","average":{"1min":1.1899999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.22900000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":5.298,"5min":10.33,"15min":15.173},"last":0.27200000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.96499999999999997,"15min":0.99199999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.6859999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.308},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.96799999999999997,"15min":0.81100000000000005},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.4309999999999992,"5min":10.308,"15min":10.308},"last":0.435}]}]},{"osd":0,"up_from":8,"seq":34359739399,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7850456,"kb_used_data":5790220,"kb_used_omap":718,"kb_used_meta":2059505,"kb_avail":86521384,"statfs":{"total":96636764160,"available":88597897216,"internally_reserved":0,"allocated":5929185280,"data_stored":8938069184,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":2108933727},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,3,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.3500000000000001,"15min":1.073},"min":{"1min":0.23899999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":6.335,"5min":10.897,"15min":19.274000000000001},"last":1.288},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.95499999999999996,"15min":0.876},"min":{"1min":0.22,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.4169999999999998,"5min":10.669,"15min":19.376000000000001},"last":1.329}]},{"osd":2,"last update":"Sat Oct 14 01:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.626,"15min":0.85799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20200000000000001},"max":{"1min":4.532,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.248},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.73299999999999998,"15min":0.78600000000000003},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":2.4580000000000002,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.30099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5928497152,"data_stored":8937535347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9725317120,"data_stored":14663359532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3798089728,"data_stored":5727093945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:25:48.393 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:48.393 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:48.394 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:48.394 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:48.394 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:25:48.395 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:25:52.991 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:52.986+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:25:53.336 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:53.334+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:25:53.341 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:53.338+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:25:53.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:53.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.498+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:53.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.498+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:53.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.498+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:53.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.770+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:25:53.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:53.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:53.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:53.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:25:53.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.814+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:25:53.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:53.818+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:58.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:58.118+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:25:58.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:58.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:58.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:58.146+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:58.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:58.146+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:25:58.653 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:25:58.650+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:25:59.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:59.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:25:59.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:59.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:25:59.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:59.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:25:59.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:59.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:25:59.763 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:25:59.762+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:01.594 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:01.590+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:07.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.030+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:07.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:07.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.054+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:07.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:07.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:07.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.058+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:07.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.062+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:07.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.062+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:07.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.410+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:07.436 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:07.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:07.490+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:08.396 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:26:08.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:08.690+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:09.176 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:26:09.177 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:26:09.242 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2731,"stamp":"2023-10-14T01:26:08.912768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28866866764,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2418221,"num_read_kb":9440295,"num_write":2382956,"num_write_kb":9531613,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80648,"ondisk_log_size":80648,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23687996,"kb_used_data":19061092,"kb_used_omap":2476,"kb_used_meta":4624403,"kb_avail":259427524,"statfs":{"total":289910292480,"available":265653784576,"internally_reserved":0,"allocated":19518558208,"data_stored":29393647135,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2535877,"internal_metadata":4735389243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,3,7,10,6,6,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16146432,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3965,"num_read_kb":15832,"num_write":3950,"num_write_kb":15800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002226"},"pg_stats":[{"pgid":"2.7","version":"137'279110","reported_seq":555848,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:04.491931+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:26:04.491931+0000","last_peered":"2023-10-14T01:26:04.491931+0000","last_clean":"2023-10-14T01:26:04.491931+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:26:04.491931+0000","last_undegraded":"2023-10-14T01:26:04.491931+0000","last_fullsized":"2023-10-14T01:26:04.491931+0000","mapping_epoch":13,"log_start":"134'269011","ondisk_log_start":"134'269011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3359813815,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":277574,"num_read_kb":1095068,"num_write":278054,"num_write_kb":1112201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"137'293898","reported_seq":580189,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:04.360184+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:26:04.360184+0000","last_peered":"2023-10-14T01:26:04.360184+0000","last_clean":"2023-10-14T01:26:04.360184+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:26:04.360184+0000","last_undegraded":"2023-10-14T01:26:04.360184+0000","last_fullsized":"2023-10-14T01:26:04.360184+0000","mapping_epoch":13,"log_start":"134'283808","ondisk_log_start":"134'283808","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3387891979,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":288270,"num_read_kb":1115341,"num_write":279383,"num_write_kb":1117131,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"137'300741","reported_seq":596787,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:04.408163+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:26:04.408163+0000","last_peered":"2023-10-14T01:26:04.408163+0000","last_clean":"2023-10-14T01:26:04.408163+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:26:04.408163+0000","last_undegraded":"2023-10-14T01:26:04.408163+0000","last_fullsized":"2023-10-14T01:26:04.408163+0000","mapping_epoch":13,"log_start":"134'290714","ondisk_log_start":"134'290714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 604s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3560907307,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":296613,"num_read_kb":1166486,"num_write":293992,"num_write_kb":1175847,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"137'318232","reported_seq":625217,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:04.380362+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:26:04.380362+0000","last_peered":"2023-10-14T01:26:04.380362+0000","last_clean":"2023-10-14T01:26:04.380362+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:26:04.380362+0000","last_undegraded":"2023-10-14T01:26:04.380362+0000","last_fullsized":"2023-10-14T01:26:04.380362+0000","mapping_epoch":13,"log_start":"134'308213","ondisk_log_start":"134'308213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1713,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 677s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3659977135,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":307536,"num_read_kb":1196631,"num_write":302653,"num_write_kb":1210294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"137'305756","reported_seq":611418,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:05.217562+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:26:05.217562+0000","last_peered":"2023-10-14T01:26:05.217562+0000","last_clean":"2023-10-14T01:26:05.217562+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:26:05.217562+0000","last_undegraded":"2023-10-14T01:26:05.217562+0000","last_fullsized":"2023-10-14T01:26:05.217562+0000","mapping_epoch":13,"log_start":"134'295712","ondisk_log_start":"134'295712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3605893591,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":305618,"num_read_kb":1181927,"num_write":298228,"num_write_kb":1192405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"137'321206","reported_seq":638974,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:04.287670+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:26:04.287670+0000","last_peered":"2023-10-14T01:26:04.287670+0000","last_clean":"2023-10-14T01:26:04.287670+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:26:04.287670+0000","last_undegraded":"2023-10-14T01:26:04.287670+0000","last_fullsized":"2023-10-14T01:26:04.287670+0000","mapping_epoch":13,"log_start":"134'311112","ondisk_log_start":"134'311112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3827306916,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":318160,"num_read_kb":1246704,"num_write":314679,"num_write_kb":1258542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"137'318115","reported_seq":631094,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:04.212671+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:26:04.212671+0000","last_peered":"2023-10-14T01:26:04.212671+0000","last_clean":"2023-10-14T01:26:04.212671+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:26:04.212671+0000","last_undegraded":"2023-10-14T01:26:04.212671+0000","last_fullsized":"2023-10-14T01:26:04.212671+0000","mapping_epoch":13,"log_start":"134'308015","ondisk_log_start":"134'308015","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3776364897,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":313430,"num_read_kb":1228554,"num_write":311587,"num_write_kb":1246158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"137'323965","reported_seq":634498,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:04.329668+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:26:04.329668+0000","last_peered":"2023-10-14T01:26:04.329668+0000","last_clean":"2023-10-14T01:26:04.329668+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:26:04.329668+0000","last_undegraded":"2023-10-14T01:26:04.329668+0000","last_fullsized":"2023-10-14T01:26:04.329668+0000","mapping_epoch":13,"log_start":"134'313917","ondisk_log_start":"134'313917","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1387,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 623s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3688120756,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":310894,"num_read_kb":1209475,"num_write":304211,"num_write_kb":1216487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":349,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:33.778997+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:25:33.778997+0000","last_peered":"2023-10-14T01:25:33.778997+0000","last_clean":"2023-10-14T01:25:33.778997+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:25:33.778997+0000","last_undegraded":"2023-10-14T01:25:33.778997+0000","last_fullsized":"2023-10-14T01:25:33.778997+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28866276396,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2418095,"num_read_kb":9440186,"num_write":2382787,"num_write_kb":9529065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19516067840,"data_stored":29392152664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80521,"ondisk_log_size":80521,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739401,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5222720,"kb_used_data":3722232,"kb_used_omap":486,"kb_used_meta":1499993,"kb_avail":89149120,"statfs":{"total":96636764160,"available":91288698880,"internally_reserved":0,"allocated":3811565568,"data_stored":5739882470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1535993524},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.23400000000000001,"5min":0.182,"15min":0.182},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.022,"15min":0.999},"min":{"1min":0.21199999999999999,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":6.7039999999999997,"5min":11.481999999999999,"15min":11.481999999999999},"last":0.41699999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:25:59 2023","interfaces":[{"interface":"back","average":{"1min":2.7839999999999998,"5min":1.9730000000000001,"15min":2.113},"min":{"1min":0.25900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":44.308},"last":0.24099999999999999},{"interface":"front","average":{"1min":2.7330000000000001,"5min":2.0150000000000001,"15min":2.1720000000000002},"min":{"1min":0.318,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":34.192999999999998,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.29999999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739391,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10560840,"kb_used_data":9529984,"kb_used_omap":1271,"kb_used_meta":1029576,"kb_avail":83811000,"statfs":{"total":96636764160,"available":85822464000,"internally_reserved":0,"allocated":9758703616,"data_stored":14696591737,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":1054286120},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,9,5,6,5],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.59999999999999998,"15min":0.73599999999999999},"min":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.18099999999999999},"max":{"1min":4.4939999999999998,"5min":7.2549999999999999,"15min":15.132999999999999},"last":2.8610000000000002},{"interface":"front","average":{"1min":1.1899999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.22900000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":5.298,"5min":10.33,"15min":15.173},"last":1.6379999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.96499999999999997,"15min":0.99199999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.6859999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.34499999999999997},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.96799999999999997,"15min":0.81100000000000005},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.4309999999999992,"5min":10.308,"15min":10.308},"last":2.645}]}]},{"osd":0,"up_from":8,"seq":34359739403,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7904436,"kb_used_data":5808876,"kb_used_omap":718,"kb_used_meta":2094833,"kb_avail":86467404,"statfs":{"total":96636764160,"available":88542621696,"internally_reserved":0,"allocated":5948289024,"data_stored":8957172928,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":735649,"internal_metadata":2145109599},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.3500000000000001,"15min":1.073},"min":{"1min":0.23899999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":6.335,"5min":10.897,"15min":19.274000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.95499999999999996,"15min":0.876},"min":{"1min":0.22,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.4169999999999998,"5min":10.669,"15min":19.376000000000001},"last":0.435}]},{"osd":2,"last update":"Sat Oct 14 01:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.626,"15min":0.85799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20200000000000001},"max":{"1min":4.532,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.73299999999999998,"15min":0.78600000000000003},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":2.4580000000000002,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.32100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5947826176,"data_stored":8956864371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9757736960,"data_stored":14695779372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3810504704,"data_stored":5739508921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:26:09.244 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:09.244 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:09.245 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:09.245 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:09.246 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:09.246 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:26:10.993 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:10.990+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:11.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:11.654+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:11.688 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:11.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:11.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:11.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:11.699 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:11.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:11.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:11.694+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:12.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.490+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:12.539 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.530+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:12.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:12.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:12.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.550+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:12.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:12.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.594+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:12.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.594+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:12.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.594+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:12.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:12.994+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:13.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:13.006+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:13.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:13.682+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:13.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:13.694+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:13.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:13.702+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:13.710 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:13.706+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:13.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:13.718+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:13.742 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:13.738+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:26:13.748 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:13.746+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:26:13.748 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:13.746+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:15.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.786+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:15.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:15.812 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:15.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:15.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.818+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:15.830 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.818+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:15.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.942+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:15.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.946+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:15.998 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:15.990+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:16.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:16.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:16.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:16.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:16.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:16.006+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:16.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:16.006+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:16.016 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:16.014+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:19.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:19.661+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:19.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:19.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:19.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:19.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:19.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:19.673+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:19.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:19.673+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:19.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:19.693+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:20.554 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:20.549+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:26:20.574 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:20.569+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:26:21.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.945+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:21.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:21.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:21.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:21.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:21.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.965+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:21.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.965+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:21.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.965+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:21.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:21.965+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:22.604 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:22.601+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:26:22.614 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:22.609+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:23.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.409+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:23.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:23.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:23.444 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:23.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:23.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:23.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.453+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:23.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:23.453+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:24.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:24.477+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:24.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:24.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:24.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:24.489+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:24.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:24.489+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:27.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.237+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:27.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:27.279 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:27.291 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:27.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.285+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:27.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.285+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:27.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:27.505+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:26:27.522 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:27.517+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:27.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:27.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:27.793+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:29.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:26:29.992 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:26:29.992 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:26:30.024 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2741,"stamp":"2023-10-14T01:26:28.916025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28899901004,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2426274,"num_read_kb":9472455,"num_write":2391040,"num_write_kb":9563949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80521,"ondisk_log_size":80521,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23412140,"kb_used_data":19125076,"kb_used_omap":2478,"kb_used_meta":4284561,"kb_avail":259703380,"statfs":{"total":289910292480,"available":265936261120,"internally_reserved":0,"allocated":19584077824,"data_stored":29459166751,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2537557,"internal_metadata":4387391403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,6,11,32,6,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16760832,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4048,"num_read_kb":16168,"num_write":4106,"num_write_kb":16424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001917"},"pg_stats":[{"pgid":"2.7","version":"137'280009","reported_seq":557658,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:25.235913+0000","last_change":"2023-10-14T00:16:06.093064+0000","last_active":"2023-10-14T01:26:25.235913+0000","last_peered":"2023-10-14T01:26:25.235913+0000","last_clean":"2023-10-14T01:26:25.235913+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:26:25.235913+0000","last_undegraded":"2023-10-14T01:26:25.235913+0000","last_fullsized":"2023-10-14T01:26:25.235913+0000","mapping_epoch":13,"log_start":"134'269911","ondisk_log_start":"134'269911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3363487927,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":278485,"num_read_kb":1098704,"num_write":278953,"num_write_kb":1115797,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"137'294954","reported_seq":582260,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:25.238821+0000","last_change":"2023-10-14T00:16:06.092495+0000","last_active":"2023-10-14T01:26:25.238821+0000","last_peered":"2023-10-14T01:26:25.238821+0000","last_clean":"2023-10-14T01:26:25.238821+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:26:25.238821+0000","last_undegraded":"2023-10-14T01:26:25.238821+0000","last_fullsized":"2023-10-14T01:26:25.238821+0000","mapping_epoch":13,"log_start":"134'284908","ondisk_log_start":"134'284908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3392053515,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":289272,"num_read_kb":1119341,"num_write":280400,"num_write_kb":1121199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"137'301738","reported_seq":598809,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:25.246117+0000","last_change":"2023-10-14T01:15:59.680639+0000","last_active":"2023-10-14T01:26:25.246117+0000","last_peered":"2023-10-14T01:26:25.246117+0000","last_clean":"2023-10-14T01:26:25.246117+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:26:25.246117+0000","last_undegraded":"2023-10-14T01:26:25.246117+0000","last_fullsized":"2023-10-14T01:26:25.246117+0000","mapping_epoch":13,"log_start":"134'291714","ondisk_log_start":"134'291714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1711,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 625s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3564773931,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":297629,"num_read_kb":1170550,"num_write":294939,"num_write_kb":1179635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"137'319329","reported_seq":627408,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:25.193087+0000","last_change":"2023-10-14T01:14:47.217815+0000","last_active":"2023-10-14T01:26:25.193087+0000","last_peered":"2023-10-14T01:26:25.193087+0000","last_clean":"2023-10-14T01:26:25.193087+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:26:25.193087+0000","last_undegraded":"2023-10-14T01:26:25.193087+0000","last_fullsized":"2023-10-14T01:26:25.193087+0000","mapping_epoch":13,"log_start":"135'309313","ondisk_log_start":"135'309313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1768,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 697s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3664138671,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":308617,"num_read_kb":1200939,"num_write":303672,"num_write_kb":1214370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"137'306749","reported_seq":613393,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:25.405435+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:26:25.405435+0000","last_peered":"2023-10-14T01:26:25.405435+0000","last_clean":"2023-10-14T01:26:25.405435+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:26:25.405435+0000","last_undegraded":"2023-10-14T01:26:25.405435+0000","last_fullsized":"2023-10-14T01:26:25.405435+0000","mapping_epoch":13,"log_start":"134'296712","ondisk_log_start":"134'296712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3609858519,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":306579,"num_read_kb":1185767,"num_write":299201,"num_write_kb":1196297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"137'322355","reported_seq":641160,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:24.516087+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:26:24.516087+0000","last_peered":"2023-10-14T01:26:24.516087+0000","last_clean":"2023-10-14T01:26:24.516087+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:26:24.516087+0000","last_undegraded":"2023-10-14T01:26:24.516087+0000","last_fullsized":"2023-10-14T01:26:24.516087+0000","mapping_epoch":13,"log_start":"135'312312","ondisk_log_start":"135'312312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3831964068,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":319193,"num_read_kb":1250832,"num_write":315818,"num_write_kb":1263098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"137'319205","reported_seq":633209,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:24.491076+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:26:24.491076+0000","last_peered":"2023-10-14T01:26:24.491076+0000","last_clean":"2023-10-14T01:26:24.491076+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:26:24.491076+0000","last_undegraded":"2023-10-14T01:26:24.491076+0000","last_fullsized":"2023-10-14T01:26:24.491076+0000","mapping_epoch":13,"log_start":"135'309115","ondisk_log_start":"135'309115","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3780788577,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":314445,"num_read_kb":1232610,"num_write":312667,"num_write_kb":1250478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"137'325057","reported_seq":636640,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:25.157113+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:26:25.157113+0000","last_peered":"2023-10-14T01:26:25.157113+0000","last_clean":"2023-10-14T01:26:25.157113+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:26:25.157113+0000","last_undegraded":"2023-10-14T01:26:25.157113+0000","last_fullsized":"2023-10-14T01:26:25.157113+0000","mapping_epoch":13,"log_start":"135'315017","ondisk_log_start":"135'315017","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1432,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 644s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3692245428,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":311928,"num_read_kb":1213603,"num_write":305221,"num_write_kb":1220527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":349,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:33.778997+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:25:33.778997+0000","last_peered":"2023-10-14T01:25:33.778997+0000","last_clean":"2023-10-14T01:25:33.778997+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:25:33.778997+0000","last_undegraded":"2023-10-14T01:25:33.778997+0000","last_fullsized":"2023-10-14T01:25:33.778997+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28899310636,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2426148,"num_read_kb":9472346,"num_write":2390871,"num_write_kb":9561401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":7017,"num_omap_keys":115,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19582115840,"data_stored":29458200664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80394,"ondisk_log_size":80394,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739405,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5308428,"kb_used_data":3734916,"kb_used_omap":486,"kb_used_meta":1573017,"kb_avail":89063412,"statfs":{"total":96636764160,"available":91200933888,"internally_reserved":0,"allocated":3824553984,"data_stored":5752870886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1610770100},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,2,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.23400000000000001,"5min":0.182,"15min":0.182},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.022,"15min":0.999},"min":{"1min":0.21199999999999999,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":6.7039999999999997,"5min":11.481999999999999,"15min":11.481999999999999},"last":0.28100000000000003}]},{"osd":1,"last update":"Sat Oct 14 01:25:59 2023","interfaces":[{"interface":"back","average":{"1min":2.7839999999999998,"5min":1.9730000000000001,"15min":2.113},"min":{"1min":0.25900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":44.308},"last":4.6929999999999996},{"interface":"front","average":{"1min":2.7330000000000001,"5min":2.0150000000000001,"15min":2.1720000000000002},"min":{"1min":0.318,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":34.192999999999998,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.40000000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739395,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10734676,"kb_used_data":9561740,"kb_used_omap":1271,"kb_used_meta":1171656,"kb_avail":83637164,"statfs":{"total":96636764160,"available":85644455936,"internally_reserved":0,"allocated":9791221760,"data_stored":14729109881,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":1199776040},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,7,30,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.59999999999999998,"15min":0.73599999999999999},"min":{"1min":0.21099999999999999,"5min":0.20999999999999999,"15min":0.18099999999999999},"max":{"1min":4.4939999999999998,"5min":7.2549999999999999,"15min":15.132999999999999},"last":4.9100000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.22900000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":5.298,"5min":10.33,"15min":15.173},"last":4.8540000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.96499999999999997,"15min":0.99199999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.6859999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":5.056},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.96799999999999997,"15min":0.81100000000000005},"min":{"1min":0.25800000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":9.4309999999999992,"5min":10.308,"15min":10.308},"last":0.379}]}]},{"osd":0,"up_from":8,"seq":34359739407,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7369036,"kb_used_data":5828420,"kb_used_omap":720,"kb_used_meta":1539887,"kb_avail":87002804,"statfs":{"total":96636764160,"available":89090871296,"internally_reserved":0,"allocated":5968302080,"data_stored":8977185984,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":737329,"internal_metadata":1576845263},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.3500000000000001,"15min":1.073},"min":{"1min":0.23899999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":6.335,"5min":10.897,"15min":19.274000000000001},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.95499999999999996,"15min":0.876},"min":{"1min":0.22,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.4169999999999998,"5min":10.669,"15min":19.376000000000001},"last":0.32200000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.626,"15min":0.85799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20200000000000001},"max":{"1min":4.532,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.31},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.73299999999999998,"15min":0.78600000000000003},"min":{"1min":0.252,"5min":0.23999999999999999,"15min":0.214},"max":{"1min":2.4580000000000002,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.46400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5967286272,"data_stored":8976324467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9791213568,"data_stored":14729255980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3823616000,"data_stored":5752620217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:26:30.026 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:30.027 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:30.027 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:30.028 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:30.028 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:30.029 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:26:31.928 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:31.925+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:26:31.933 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:31.929+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:26:31.933 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:31.929+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:32.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:32.857+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:32.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:32.889+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:32.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:32.981+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:33.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:33.001+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:34.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:34.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:34.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:34.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.397+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:34.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:34.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:34.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:34.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.409+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:34.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:34.957+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:38.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:38.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:38.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:38.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:38.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:38.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:38.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:38.201+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:39.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:39.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:39.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:39.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:39.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:39.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:39.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:39.565+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:41.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:41.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:42.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.677+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:42.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:42.761 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:42.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:42.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:42.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.769+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:42.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.769+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:42.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:42.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:43.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:43.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:43.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:43.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:43.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:43.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:43.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:43.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:43.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:43.761+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:43.768 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:43.761+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:43.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:43.777+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:44.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:44.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:44.682 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:44.677+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:44.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:44.697+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:44.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:44.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:44.721 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:44.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:44.722 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:44.713+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:47.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:47.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:47.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:47.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:47.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:47.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:47.389 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:47.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:47.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:47.385+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:47.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:47.385+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:49.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.013+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:49.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:49.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:49.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:49.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.073+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:49.921 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.913+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:49.922 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:49.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.917+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:49.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:49.945 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.941+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:49.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:49.969 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:49.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:49.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:50.030 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:26:50.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.033+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:50.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.053+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:50.093 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:50.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:50.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:50.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:50.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.165+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:50.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.165+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:50.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:50.173+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:26:50.759 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:26:50.828 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2751,"stamp":"2023-10-14T01:26:48.919227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28930883148,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2433392,"num_read_kb":9500859,"num_write":2398605,"num_write_kb":9594209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80485,"ondisk_log_size":80485,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23721208,"kb_used_data":19185112,"kb_used_omap":2478,"kb_used_meta":4533585,"kb_avail":259394312,"statfs":{"total":289910292480,"available":265619775488,"internally_reserved":0,"allocated":19645554688,"data_stored":29520643615,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2537557,"internal_metadata":4642391979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,18,16,12,5,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15986688,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3683,"num_read_kb":14688,"num_write":3908,"num_write_kb":15632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001930"},"pg_stats":[{"pgid":"2.7","version":"137'280904","reported_seq":559383,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:46.037105+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:26:46.037105+0000","last_peered":"2023-10-14T01:26:46.037105+0000","last_clean":"2023-10-14T01:26:46.037105+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:26:46.037105+0000","last_undegraded":"2023-10-14T01:26:46.037105+0000","last_fullsized":"2023-10-14T01:26:46.037105+0000","mapping_epoch":13,"log_start":"134'270811","ondisk_log_start":"134'270811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":22,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 11s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3367088311,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":279314,"num_read_kb":1102020,"num_write":279832,"num_write_kb":1119313,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"137'295918","reported_seq":584097,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:46.040628+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:26:46.040628+0000","last_peered":"2023-10-14T01:26:46.040628+0000","last_clean":"2023-10-14T01:26:46.040628+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:26:46.040628+0000","last_undegraded":"2023-10-14T01:26:46.040628+0000","last_fullsized":"2023-10-14T01:26:46.040628+0000","mapping_epoch":13,"log_start":"135'285908","ondisk_log_start":"135'285908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":64,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 16s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3395690763,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":290127,"num_read_kb":1122753,"num_write":281287,"num_write_kb":1124747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"137'302713","reported_seq":600702,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:46.033653+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:26:46.033653+0000","last_peered":"2023-10-14T01:26:46.033653+0000","last_clean":"2023-10-14T01:26:46.033653+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:26:46.033653+0000","last_undegraded":"2023-10-14T01:26:46.033653+0000","last_fullsized":"2023-10-14T01:26:46.033653+0000","mapping_epoch":13,"log_start":"135'292614","ondisk_log_start":"135'292614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3568611883,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":298531,"num_read_kb":1174146,"num_write":295877,"num_write_kb":1183387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"137'320353","reported_seq":629399,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:45.986245+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:26:45.986245+0000","last_peered":"2023-10-14T01:26:45.986245+0000","last_clean":"2023-10-14T01:26:45.986245+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:26:45.986245+0000","last_undegraded":"2023-10-14T01:26:45.986245+0000","last_fullsized":"2023-10-14T01:26:45.986245+0000","mapping_epoch":13,"log_start":"135'310313","ondisk_log_start":"135'310313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3668148655,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":309568,"num_read_kb":1204727,"num_write":304653,"num_write_kb":1218294,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"137'307655","reported_seq":615183,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:45.509293+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:26:45.509293+0000","last_peered":"2023-10-14T01:26:45.509293+0000","last_clean":"2023-10-14T01:26:45.509293+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:26:45.509293+0000","last_undegraded":"2023-10-14T01:26:45.509293+0000","last_fullsized":"2023-10-14T01:26:45.509293+0000","mapping_epoch":13,"log_start":"135'297612","ondisk_log_start":"135'297612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3613577687,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":307443,"num_read_kb":1189215,"num_write":300107,"num_write_kb":1199921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"137'323343","reported_seq":643051,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:44.815331+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:26:44.815331+0000","last_peered":"2023-10-14T01:26:44.815331+0000","last_clean":"2023-10-14T01:26:44.815331+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:26:44.815331+0000","last_undegraded":"2023-10-14T01:26:44.815331+0000","last_fullsized":"2023-10-14T01:26:44.815331+0000","mapping_epoch":13,"log_start":"135'313312","ondisk_log_start":"135'313312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3835990436,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":320087,"num_read_kb":1254396,"num_write":316800,"num_write_kb":1267026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"137'320231","reported_seq":635151,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:26:44.848767+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:26:44.848767+0000","last_peered":"2023-10-14T01:26:44.848767+0000","last_clean":"2023-10-14T01:26:44.848767+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:26:44.848767+0000","last_undegraded":"2023-10-14T01:26:44.848767+0000","last_fullsized":"2023-10-14T01:26:44.848767+0000","mapping_epoch":13,"log_start":"135'310215","ondisk_log_start":"135'310215","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3784966497,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":315353,"num_read_kb":1236238,"num_write":313687,"num_write_kb":1254558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"137'326143","reported_seq":638671,"reported_epoch":137,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:26:45.995882+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:26:45.995882+0000","last_peered":"2023-10-14T01:26:45.995882+0000","last_clean":"2023-10-14T01:26:45.995882+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:26:45.995882+0000","last_undegraded":"2023-10-14T01:26:45.995882+0000","last_fullsized":"2023-10-14T01:26:45.995882+0000","mapping_epoch":13,"log_start":"135'316117","ondisk_log_start":"135'316117","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1541,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 665s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3696218548,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":312843,"num_read_kb":1217255,"num_write":306193,"num_write_kb":1224415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":349,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:33.778997+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:25:33.778997+0000","last_peered":"2023-10-14T01:25:33.778997+0000","last_clean":"2023-10-14T01:25:33.778997+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:25:33.778997+0000","last_undegraded":"2023-10-14T01:25:33.778997+0000","last_fullsized":"2023-10-14T01:25:33.778997+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28930292780,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2433266,"num_read_kb":9500750,"num_write":2398436,"num_write_kb":9591661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19643760640,"data_stored":29519845464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80358,"ondisk_log_size":80358,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739409,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5352008,"kb_used_data":3746560,"kb_used_omap":486,"kb_used_meta":1604953,"kb_avail":89019832,"statfs":{"total":96636764160,"available":91156307968,"internally_reserved":0,"allocated":3836477440,"data_stored":5764794342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1643472564},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.57499999999999996,"15min":0.63700000000000001},"min":{"1min":0.23400000000000001,"5min":0.182,"15min":0.182},"max":{"1min":4.4569999999999999,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":1.022,"15min":0.999},"min":{"1min":0.21199999999999999,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":6.7039999999999997,"5min":11.481999999999999,"15min":11.481999999999999},"last":0.23999999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:25:59 2023","interfaces":[{"interface":"back","average":{"1min":2.7839999999999998,"5min":1.9730000000000001,"15min":2.113},"min":{"1min":0.25900000000000001,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":34.158000000000001,"5min":34.158000000000001,"15min":44.308},"last":0.50800000000000001},{"interface":"front","average":{"1min":2.7330000000000001,"5min":2.0150000000000001,"15min":2.1720000000000002},"min":{"1min":0.318,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":34.192999999999998,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.36399999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739399,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10883376,"kb_used_data":9592420,"kb_used_omap":1271,"kb_used_meta":1289672,"kb_avail":83488464,"statfs":{"total":96636764160,"available":85492187136,"internally_reserved":0,"allocated":9822638080,"data_stored":14760526201,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":1320624424},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,12,12,7,5,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:26:26 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":6.4050000000000002,"5min":7.2549999999999999,"15min":15.132999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88800000000000001,"15min":0.95299999999999996},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.8540000000000001,"5min":10.33,"15min":15.173},"last":0.45900000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.27100000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.056,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0129999999999999,"15min":0.80200000000000005},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":5.3079999999999998,"5min":10.308,"15min":10.308},"last":0.77500000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739411,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7485824,"kb_used_data":5846132,"kb_used_omap":720,"kb_used_meta":1638959,"kb_avail":86886016,"statfs":{"total":96636764160,"available":88971280384,"internally_reserved":0,"allocated":5986439168,"data_stored":8995323072,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":737329,"internal_metadata":1678294991},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,4,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:26:26 2023","interfaces":[{"interface":"back","average":{"1min":2.008,"5min":1.4330000000000001,"15min":1.173},"min":{"1min":0.23699999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":9.0830000000000002,"5min":10.897,"15min":19.274000000000001},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.0449999999999999,"15min":0.93700000000000006},"min":{"1min":0.26800000000000002,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":4.806,"5min":10.669,"15min":19.376000000000001},"last":0.25700000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65500000000000003,"15min":0.85899999999999999},"min":{"1min":0.214,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":3.1219999999999999,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.78000000000000003,"15min":0.82099999999999995},"min":{"1min":0.27200000000000002,"5min":0.249,"15min":0.214},"max":{"1min":4.508,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.35699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5985517568,"data_stored":8994555763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9822535680,"data_stored":14760578092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3835707392,"data_stored":5764711609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:26:50.830 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:50.830 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:50.831 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:26:50.831 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:26:51.522 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:51.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:51.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:51.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:51.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:51.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:51.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:51.569+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:52.993 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:52.985+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:26:53.008 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:53.005+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:54.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.385+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:54.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:54.442 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:54.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:54.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:54.463 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.453+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:54.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.453+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:54.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.461+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:54.491 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:54.485+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:26:54.522 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:54.517+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:54.671 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.665+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:26:54.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:54.694 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:54.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:54.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.685+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:54.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.685+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:54.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.685+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:54.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:54.685+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:54.852 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:54.849+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:26:54.858 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:54.853+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:55.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.084+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:55.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:55.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:26:55.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:26:55.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.124+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:26:55.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.124+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:26:55.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.144+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:55.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:55.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:55.567 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.556+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:26:55.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:55.572+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:57.158 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:26:57.152+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:26:57.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:57.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:57.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:57.620+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:57.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:57.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:26:59.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:59.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:26:59.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:59.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:26:59.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:26:59.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:01.545 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:01.540+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:27:01.551 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:01.544+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:02.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:02.852+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:27:02.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:02.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:02.873 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:02.868+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:02.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:02.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:02.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:02.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:02.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:02.884+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:03.498 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:03.488+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:27:03.499 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:03.488+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:03.589 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:03.584+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:04.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:04.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.384+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:04.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:04.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:04.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.408+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:04.414 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.408+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:04.415 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.408+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:04.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:04.432+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:05.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:05.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:05.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:05.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.092+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:05.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.132+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:27:05.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.140+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:05.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:05.185 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.180+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:05.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:05.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:06.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.908+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:27:06.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.908+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:27:06.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:06.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:06.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:06.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:06.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:06.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:06.928+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:07.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:07.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:07.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:07.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.244+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:07.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.244+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:07.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.932+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:27:07.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.932+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:27:07.944 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.940+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:07.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:07.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:07.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:08.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:08.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:08.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:08.000+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:08.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:08.000+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:08.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:08.028+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:10.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.548+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:27:10.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:10.587 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:27:10.580+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:27:10.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:10.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:10.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:10.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:10.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.632+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:10.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.632+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:10.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.648+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:10.832 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:27:10.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.896+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:10.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:10.942 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:10.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:10.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.936+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:10.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.936+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:10.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:10.952+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:27:11.707 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:27:11.732 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2763,"stamp":"2023-10-14T01:27:10.927817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28966710860,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2441954,"num_read_kb":9535027,"num_write":2407354,"num_write_kb":9629205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80408,"ondisk_log_size":80408,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24036508,"kb_used_data":19256064,"kb_used_omap":2478,"kb_used_meta":4777937,"kb_avail":259079012,"statfs":{"total":289910292480,"available":265296908288,"internally_reserved":0,"allocated":19718209536,"data_stored":29593284573,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2537557,"internal_metadata":4892608427},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,10,12,17,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19836928,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4737,"num_read_kb":18896,"num_write":4842,"num_write_kb":19368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006902"},"pg_stats":[{"pgid":"2.7","version":"138'281920","reported_seq":561336,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:06.889260+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:27:06.889260+0000","last_peered":"2023-10-14T01:27:06.889260+0000","last_clean":"2023-10-14T01:27:06.889260+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:27:06.889260+0000","last_undegraded":"2023-10-14T01:27:06.889260+0000","last_fullsized":"2023-10-14T01:27:06.889260+0000","mapping_epoch":13,"log_start":"135'271911","ondisk_log_start":"135'271911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":82,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 32s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3371069623,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":280237,"num_read_kb":1105708,"num_write":280804,"num_write_kb":1123201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"138'296913","reported_seq":586080,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:06.912634+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:27:06.912634+0000","last_peered":"2023-10-14T01:27:06.912634+0000","last_clean":"2023-10-14T01:27:06.912634+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:27:06.912634+0000","last_undegraded":"2023-10-14T01:27:06.912634+0000","last_fullsized":"2023-10-14T01:27:06.912634+0000","mapping_epoch":13,"log_start":"135'286908","ondisk_log_start":"135'286908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":152,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 37s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3399348491,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":291090,"num_read_kb":1126597,"num_write":282181,"num_write_kb":1128323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"138'303727","reported_seq":602687,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:06.900852+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:27:06.900852+0000","last_peered":"2023-10-14T01:27:06.900852+0000","last_clean":"2023-10-14T01:27:06.900852+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:27:06.900852+0000","last_undegraded":"2023-10-14T01:27:06.900852+0000","last_fullsized":"2023-10-14T01:27:06.900852+0000","mapping_epoch":13,"log_start":"135'293714","ondisk_log_start":"135'293714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3572703787,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":299496,"num_read_kb":1177998,"num_write":296875,"num_write_kb":1187379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"138'321485","reported_seq":631581,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:06.901354+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:27:06.901354+0000","last_peered":"2023-10-14T01:27:06.901354+0000","last_clean":"2023-10-14T01:27:06.901354+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:27:06.901354+0000","last_undegraded":"2023-10-14T01:27:06.901354+0000","last_fullsized":"2023-10-14T01:27:06.901354+0000","mapping_epoch":13,"log_start":"135'311413","ondisk_log_start":"135'311413","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3672580527,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":310607,"num_read_kb":1208867,"num_write":305737,"num_write_kb":1222630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"138'308913","reported_seq":617636,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:10.715476+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:27:10.715476+0000","last_peered":"2023-10-14T01:27:10.715476+0000","last_clean":"2023-10-14T01:27:10.715476+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:27:10.715476+0000","last_undegraded":"2023-10-14T01:27:10.715476+0000","last_fullsized":"2023-10-14T01:27:10.715476+0000","mapping_epoch":13,"log_start":"135'298912","ondisk_log_start":"135'298912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3618611671,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":308613,"num_read_kb":1193879,"num_write":301335,"num_write_kb":1204833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"138'324673","reported_seq":645660,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:10.169243+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:27:10.169243+0000","last_peered":"2023-10-14T01:27:10.169243+0000","last_clean":"2023-10-14T01:27:10.169243+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:27:10.169243+0000","last_undegraded":"2023-10-14T01:27:10.169243+0000","last_fullsized":"2023-10-14T01:27:10.169243+0000","mapping_epoch":13,"log_start":"135'314612","ondisk_log_start":"135'314612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3841372580,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":321351,"num_read_kb":1259440,"num_write":318112,"num_write_kb":1272274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"138'321504","reported_seq":637642,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:10.169875+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:27:10.169875+0000","last_peered":"2023-10-14T01:27:10.169875+0000","last_clean":"2023-10-14T01:27:10.169875+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:27:10.169875+0000","last_undegraded":"2023-10-14T01:27:10.169875+0000","last_fullsized":"2023-10-14T01:27:10.169875+0000","mapping_epoch":13,"log_start":"135'311415","ondisk_log_start":"135'311415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3790094689,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":316560,"num_read_kb":1241058,"num_write":314942,"num_write_kb":1259578,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"138'327248","reported_seq":640832,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:06.892352+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:27:06.892352+0000","last_peered":"2023-10-14T01:27:06.892352+0000","last_clean":"2023-10-14T01:27:06.892352+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:27:06.892352+0000","last_undegraded":"2023-10-14T01:27:06.892352+0000","last_fullsized":"2023-10-14T01:27:06.892352+0000","mapping_epoch":13,"log_start":"135'317217","ondisk_log_start":"135'317217","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1592,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 686s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3700339124,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":313874,"num_read_kb":1221371,"num_write":307199,"num_write_kb":1228439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":350,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:38.701986+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:25:38.701986+0000","last_peered":"2023-10-14T01:25:38.701986+0000","last_clean":"2023-10-14T01:25:38.701986+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:25:38.701986+0000","last_undegraded":"2023-10-14T01:25:38.701986+0000","last_fullsized":"2023-10-14T01:25:38.701986+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28966120492,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2441828,"num_read_kb":9534918,"num_write":2407185,"num_write_kb":9626657,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19716833280,"data_stored":29592918104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80281,"ondisk_log_size":80281,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739414,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5434968,"kb_used_data":3761488,"kb_used_omap":486,"kb_used_meta":1672985,"kb_avail":88936872,"statfs":{"total":96636764160,"available":91071356928,"internally_reserved":0,"allocated":3851763712,"data_stored":5780075984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1713137332},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.505,"15min":0.63400000000000001},"min":{"1min":0.22900000000000001,"5min":0.20100000000000001,"15min":0.182},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":1.2729999999999999,"5min":0.88600000000000001,"15min":1.024},"min":{"1min":0.19600000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":10.488,"5min":10.488,"15min":11.481999999999999},"last":0.32600000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:27:01 2023","interfaces":[{"interface":"back","average":{"1min":1.7689999999999999,"5min":1.962,"15min":2.161},"min":{"1min":0.24099999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.4909999999999997,"5min":34.158000000000001,"15min":44.308},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.8009999999999999,"15min":2.1539999999999999},"min":{"1min":0.28499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.6100000000000003,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.44}]}]},{"osd":1,"up_from":8,"seq":34359739403,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11031884,"kb_used_data":9624320,"kb_used_omap":1271,"kb_used_meta":1406280,"kb_avail":83339956,"statfs":{"total":96636764160,"available":85340114944,"internally_reserved":0,"allocated":9855303680,"data_stored":14793187171,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":1440031016},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,8,7,13,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:26:26 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":6.4050000000000002,"5min":7.2549999999999999,"15min":15.132999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88800000000000001,"15min":0.95299999999999996},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.8540000000000001,"5min":10.33,"15min":15.173},"last":0.753}]},{"osd":2,"last update":"Sat Oct 14 01:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.27100000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.056,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0129999999999999,"15min":0.80200000000000005},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":5.3079999999999998,"5min":10.308,"15min":10.308},"last":0.35899999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739416,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7569656,"kb_used_data":5870256,"kb_used_omap":720,"kb_used_meta":1698671,"kb_avail":86802184,"statfs":{"total":96636764160,"available":88885436416,"internally_reserved":0,"allocated":6011142144,"data_stored":9020021418,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":737329,"internal_metadata":1739440079},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:26:26 2023","interfaces":[{"interface":"back","average":{"1min":2.008,"5min":1.4330000000000001,"15min":1.173},"min":{"1min":0.23699999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":9.0830000000000002,"5min":10.897,"15min":19.274000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.0449999999999999,"15min":0.93700000000000006},"min":{"1min":0.26800000000000002,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":4.806,"5min":10.669,"15min":19.376000000000001},"last":0.38800000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:26:26 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.65500000000000003,"15min":0.85899999999999999},"min":{"1min":0.214,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":3.1219999999999999,"5min":7.0739999999999998,"15min":9.3309999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":0.78000000000000003,"15min":0.82099999999999995},"min":{"1min":0.27200000000000002,"5min":0.249,"15min":0.214},"max":{"1min":4.508,"5min":6.6879999999999997,"15min":7.5979999999999999},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6010155008,"data_stored":9019193203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9855492096,"data_stored":14793534508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3851186176,"data_stored":5780190393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:27:11.734 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:11.734 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:11.735 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:11.735 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:27:12.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:12.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:12.435 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:12.424+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:12.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:12.452+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:12.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:12.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:13.116 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:13.112+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:27:13.186 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:13.180+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:13.191 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:13.180+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:27:13.191 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:13.180+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:13.544 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:13.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:13.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:13.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:13.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:13.656+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:14.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.844+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:27:14.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:14.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:14.892 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:14.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:14.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:14.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.944+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:14.963 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.956+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:14.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:14.988+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:15.027 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.020+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:27:15.047 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:15.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.048+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:15.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:15.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.108+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:15.123 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.116+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:27:15.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.116+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:27:15.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.128+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:27:15.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:15.140 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:15.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.156+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:15.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.156+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:15.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:15.181 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:15.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:15.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:15.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:16.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:16.240+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:16.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:16.256+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:16.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:16.272+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:19.716 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:19.712+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:27:19.722 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:19.716+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:22.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.196+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:22.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:22.218 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:22.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:22.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:22.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.248+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:22.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.248+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:22.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:22.268+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:22.971 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:22.968+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:27:22.997 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:22.992+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:27.855 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:27.852+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:27.857 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:27.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:31.736 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:27:32.670 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:27:32.671 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:27:32.712 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2773,"stamp":"2023-10-14T01:27:30.936111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":28998671948,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2449783,"num_read_kb":9566283,"num_write":2415170,"num_write_kb":9660469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80522,"ondisk_log_size":80522,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24282596,"kb_used_data":19319880,"kb_used_omap":2478,"kb_used_meta":4960209,"kb_avail":258832924,"statfs":{"total":289910292480,"available":265044914176,"internally_reserved":0,"allocated":19783557120,"data_stored":29658632157,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2537557,"internal_metadata":5079254955},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,5,12,11,11,9,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19197952,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4635,"num_read_kb":18516,"num_write":4693,"num_write_kb":18772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001990"},"pg_stats":[{"pgid":"2.7","version":"138'282906","reported_seq":563272,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:27.626905+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:27:27.626905+0000","last_peered":"2023-10-14T01:27:27.626905+0000","last_clean":"2023-10-14T01:27:27.626905+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:27:27.626905+0000","last_undegraded":"2023-10-14T01:27:27.626905+0000","last_fullsized":"2023-10-14T01:27:27.626905+0000","mapping_epoch":13,"log_start":"135'272811","ondisk_log_start":"135'272811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":151,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 53s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3374915767,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":281176,"num_read_kb":1109464,"num_write":281744,"num_write_kb":1126961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"138'297936","reported_seq":588037,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:27.635949+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:27:27.635949+0000","last_peered":"2023-10-14T01:27:27.635949+0000","last_clean":"2023-10-14T01:27:27.635949+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:27:27.635949+0000","last_undegraded":"2023-10-14T01:27:27.635949+0000","last_fullsized":"2023-10-14T01:27:27.635949+0000","mapping_epoch":13,"log_start":"135'287908","ondisk_log_start":"135'287908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":206,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 58s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3403227403,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":292007,"num_read_kb":1130261,"num_write":283128,"num_write_kb":1132111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"138'304740","reported_seq":604672,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:27.633910+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:27:27.633910+0000","last_peered":"2023-10-14T01:27:27.633910+0000","last_clean":"2023-10-14T01:27:27.633910+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:27:27.633910+0000","last_undegraded":"2023-10-14T01:27:27.633910+0000","last_fullsized":"2023-10-14T01:27:27.633910+0000","mapping_epoch":13,"log_start":"135'294714","ondisk_log_start":"135'294714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3576799787,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":300466,"num_read_kb":1181870,"num_write":297875,"num_write_kb":1191379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"138'322590","reported_seq":633729,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:27.600082+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:27:27.600082+0000","last_peered":"2023-10-14T01:27:27.600082+0000","last_clean":"2023-10-14T01:27:27.600082+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:27:27.600082+0000","last_undegraded":"2023-10-14T01:27:27.600082+0000","last_fullsized":"2023-10-14T01:27:27.600082+0000","mapping_epoch":13,"log_start":"135'312513","ondisk_log_start":"135'312513","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3676934575,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":311634,"num_read_kb":1212967,"num_write":306803,"num_write_kb":1226894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"138'309845","reported_seq":619505,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:30.857111+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:27:30.857111+0000","last_peered":"2023-10-14T01:27:30.857111+0000","last_clean":"2023-10-14T01:27:30.857111+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:27:30.857111+0000","last_undegraded":"2023-10-14T01:27:30.857111+0000","last_fullsized":"2023-10-14T01:27:30.857111+0000","mapping_epoch":13,"log_start":"135'299812","ondisk_log_start":"135'299812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3622371799,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":309530,"num_read_kb":1197539,"num_write":302257,"num_write_kb":1208521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"138'325677","reported_seq":647722,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:30.314356+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:27:30.314356+0000","last_peered":"2023-10-14T01:27:30.314356+0000","last_clean":"2023-10-14T01:27:30.314356+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:27:30.314356+0000","last_undegraded":"2023-10-14T01:27:30.314356+0000","last_fullsized":"2023-10-14T01:27:30.314356+0000","mapping_epoch":13,"log_start":"135'315612","ondisk_log_start":"135'315612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3845419428,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":322401,"num_read_kb":1263640,"num_write":319105,"num_write_kb":1276246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"138'322478","reported_seq":639638,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:30.338889+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:27:30.338889+0000","last_peered":"2023-10-14T01:27:30.338889+0000","last_clean":"2023-10-14T01:27:30.338889+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:27:30.338889+0000","last_undegraded":"2023-10-14T01:27:30.338889+0000","last_fullsized":"2023-10-14T01:27:30.338889+0000","mapping_epoch":13,"log_start":"135'312415","ondisk_log_start":"135'312415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3794039137,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":317577,"num_read_kb":1245118,"num_write":315904,"num_write_kb":1263426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"138'328325","reported_seq":642925,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:27.605422+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:27:27.605422+0000","last_peered":"2023-10-14T01:27:27.605422+0000","last_clean":"2023-10-14T01:27:27.605422+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:27:27.605422+0000","last_undegraded":"2023-10-14T01:27:27.605422+0000","last_fullsized":"2023-10-14T01:27:27.605422+0000","mapping_epoch":13,"log_start":"135'318317","ondisk_log_start":"135'318317","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1659,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 706s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3704373684,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":314866,"num_read_kb":1225315,"num_write":308185,"num_write_kb":1232383,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":350,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-10-14T01:25:38.701986+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:25:38.701986+0000","last_peered":"2023-10-14T01:25:38.701986+0000","last_clean":"2023-10-14T01:25:38.701986+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:25:38.701986+0000","last_undegraded":"2023-10-14T01:25:38.701986+0000","last_fullsized":"2023-10-14T01:25:38.701986+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":28998081580,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2449657,"num_read_kb":9566174,"num_write":2415001,"num_write_kb":9657921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19780730880,"data_stored":29656815704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80395,"ondisk_log_size":80395,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739418,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5478992,"kb_used_data":3773576,"kb_used_omap":486,"kb_used_meta":1704921,"kb_avail":88892848,"statfs":{"total":96636764160,"available":91026276352,"internally_reserved":0,"allocated":3864141824,"data_stored":5792454096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1745839796},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.505,"15min":0.63400000000000001},"min":{"1min":0.22900000000000001,"5min":0.20100000000000001,"15min":0.182},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":1.2729999999999999,"5min":0.88600000000000001,"15min":1.024},"min":{"1min":0.19600000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":10.488,"5min":10.488,"15min":11.481999999999999},"last":0.253}]},{"osd":1,"last update":"Sat Oct 14 01:27:01 2023","interfaces":[{"interface":"back","average":{"1min":1.7689999999999999,"5min":1.962,"15min":2.161},"min":{"1min":0.24099999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.4909999999999997,"5min":34.158000000000001,"15min":44.308},"last":0.5},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.8009999999999999,"15min":2.1539999999999999},"min":{"1min":0.28499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.6100000000000003,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.317}]}]},{"osd":1,"up_from":8,"seq":34359739407,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11138348,"kb_used_data":9656804,"kb_used_omap":1271,"kb_used_meta":1480264,"kb_avail":83233492,"statfs":{"total":96636764160,"available":85231095808,"internally_reserved":0,"allocated":9888567296,"data_stored":14826450787,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":1515790632},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1,4,8,5,6,7,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:26:26 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":0.79900000000000004,"15min":0.78000000000000003},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":6.4050000000000002,"5min":7.2549999999999999,"15min":15.132999999999999},"last":16.481000000000002},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.88800000000000001,"15min":0.95299999999999996},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":4.8540000000000001,"5min":10.33,"15min":15.173},"last":16.52}]},{"osd":2,"last update":"Sat Oct 14 01:26:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.27100000000000002,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.056,"5min":10.375999999999999,"15min":10.375999999999999},"last":4.6340000000000003},{"interface":"front","average":{"1min":0.93400000000000005,"5min":1.0129999999999999,"15min":0.80200000000000005},"min":{"1min":0.193,"5min":0.193,"15min":0.193},"max":{"1min":5.3079999999999998,"5min":10.308,"15min":10.308},"last":0.32900000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739420,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7665256,"kb_used_data":5889500,"kb_used_omap":720,"kb_used_meta":1775023,"kb_avail":86706584,"statfs":{"total":96636764160,"available":88787542016,"internally_reserved":0,"allocated":6030848000,"data_stored":9039727274,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":737329,"internal_metadata":1817624527},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":2.024,"5min":1.575,"15min":1.2689999999999999},"min":{"1min":0.20399999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":10.837999999999999,"5min":10.897,"15min":19.274000000000001},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":1.0149999999999999,"15min":0.93899999999999995},"min":{"1min":0.214,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":6.0499999999999998,"5min":10.669,"15min":19.376000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.73699999999999999,"15min":0.86199999999999999},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":7.1319999999999997,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.222,"5min":0.86799999999999999,"15min":0.85699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":7.6289999999999996,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.32900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6029721600,"data_stored":9038759795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9887887360,"data_stored":14825929772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3863121920,"data_stored":5792126137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:27:32.714 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:32.714 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:32.715 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:32.715 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:27:32.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:32.876+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:32.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:32.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:32.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:32.936+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:32.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:32.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:32.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:32.971+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:33.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.507+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:33.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.531+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:33.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:33.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:33.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:33.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.567+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:33.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.567+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:33.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:33.591+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:35.474 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:35.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:35.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:35.471+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:35.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:35.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:35.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:35.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:35.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:35.491+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:35.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:35.491+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:36.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:36.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:36.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:36.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:36.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:36.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.103+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:36.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.103+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:36.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:36.111+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:39.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:39.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.207+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:39.275 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.275+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:39.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:39.285 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.283+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:39.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.283+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:39.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.283+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:39.287 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:39.283+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:41.568 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:41.567+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:41.579 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:41.575+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:41.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:41.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:41.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:41.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:41.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:41.599+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:41.607 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:41.599+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:43.059 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:43.055+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:27:43.574 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:43.571+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:27:46.179 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:46.175+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:27:46.271 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:46.267+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:46.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:46.891+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:46.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:46.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:46.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:46.943+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:46.947 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:46.943+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:47.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:47.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:47.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:47.859+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:47.887 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:47.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:47.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:47.915+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:47.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:47.915+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:47.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:47.915+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:47.940 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:47.939+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:48.413 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.411+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:27:48.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.431+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:27:48.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:48.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.495+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:48.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:48.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.535+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:48.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:48.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.539+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:48.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.539+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:48.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:48.691+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:49.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.131+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:49.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.151+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:49.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.199+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:49.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:49.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:49.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:49.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:49.716 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:49.707+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:50.680 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:50.679+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:27:50.689 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:50.687+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:27:50.692 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:50.691+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:51.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:51.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:51.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:51.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:52.716 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:27:52.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:52.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:52.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:52.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:52.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:52.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:53.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:53.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:27:53.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:53.007+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:27:53.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:53.007+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:27:53.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:53.007+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:27:53.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:53.031+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:53.700 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:27:53.701 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:27:53.764 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2784,"stamp":"2023-10-14T01:27:52.942638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29031374412,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2457730,"num_read_kb":9597995,"num_write":2423157,"num_write_kb":9692417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80593,"ondisk_log_size":80593,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24632632,"kb_used_data":19381916,"kb_used_omap":2478,"kb_used_meta":5248209,"kb_avail":258482888,"statfs":{"total":289910292480,"available":264686477312,"internally_reserved":0,"allocated":19847081984,"data_stored":29722157021,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2537557,"internal_metadata":5374166955},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,10,20,15,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16330752,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3985,"num_read_kb":15900,"num_write":3986,"num_write_kb":15944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002024"},"pg_stats":[{"pgid":"2.7","version":"138'283897","reported_seq":565195,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:48.221867+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:27:48.221867+0000","last_peered":"2023-10-14T01:27:48.221867+0000","last_clean":"2023-10-14T01:27:48.221867+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:27:48.221867+0000","last_undegraded":"2023-10-14T01:27:48.221867+0000","last_fullsized":"2023-10-14T01:27:48.221867+0000","mapping_epoch":13,"log_start":"135'273811","ondisk_log_start":"135'273811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":213,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 73s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3378802871,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":282106,"num_read_kb":1113172,"num_write":282694,"num_write_kb":1130761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"138'298964","reported_seq":590071,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:48.194850+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:27:48.194850+0000","last_peered":"2023-10-14T01:27:48.194850+0000","last_clean":"2023-10-14T01:27:48.194850+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:27:48.194850+0000","last_undegraded":"2023-10-14T01:27:48.194850+0000","last_fullsized":"2023-10-14T01:27:48.194850+0000","mapping_epoch":13,"log_start":"135'288908","ondisk_log_start":"135'288908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":280,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 78s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3407106315,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":292989,"num_read_kb":1134181,"num_write":284073,"num_write_kb":1135891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"138'305755","reported_seq":606681,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:48.136174+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:27:48.136174+0000","last_peered":"2023-10-14T01:27:48.136174+0000","last_clean":"2023-10-14T01:27:48.136174+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:27:48.136174+0000","last_undegraded":"2023-10-14T01:27:48.136174+0000","last_fullsized":"2023-10-14T01:27:48.136174+0000","mapping_epoch":13,"log_start":"135'295714","ondisk_log_start":"135'295714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3580920363,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":301457,"num_read_kb":1185830,"num_write":298879,"num_write_kb":1195395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"138'323646","reported_seq":635753,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:48.120811+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:27:48.120811+0000","last_peered":"2023-10-14T01:27:48.120811+0000","last_clean":"2023-10-14T01:27:48.120811+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:27:48.120811+0000","last_undegraded":"2023-10-14T01:27:48.120811+0000","last_fullsized":"2023-10-14T01:27:48.120811+0000","mapping_epoch":13,"log_start":"136'313613","ondisk_log_start":"136'313613","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3681128879,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":312590,"num_read_kb":1216775,"num_write":307826,"num_write_kb":1230986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"138'310805","reported_seq":621407,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:51.010498+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:27:51.010498+0000","last_peered":"2023-10-14T01:27:51.010498+0000","last_clean":"2023-10-14T01:27:51.010498+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:27:51.010498+0000","last_undegraded":"2023-10-14T01:27:51.010498+0000","last_fullsized":"2023-10-14T01:27:51.010498+0000","mapping_epoch":13,"log_start":"136'300712","ondisk_log_start":"136'300712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3626242519,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":310450,"num_read_kb":1201203,"num_write":303197,"num_write_kb":1212281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"138'326744","reported_seq":649837,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:50.600536+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:27:50.600536+0000","last_peered":"2023-10-14T01:27:50.600536+0000","last_clean":"2023-10-14T01:27:50.600536+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:27:50.600536+0000","last_undegraded":"2023-10-14T01:27:50.600536+0000","last_fullsized":"2023-10-14T01:27:50.600536+0000","mapping_epoch":13,"log_start":"136'316712","ondisk_log_start":"136'316712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3849748900,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":323444,"num_read_kb":1267808,"num_write":320164,"num_write_kb":1280482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"138'323559","reported_seq":641781,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:50.544989+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:27:50.544989+0000","last_peered":"2023-10-14T01:27:50.544989+0000","last_clean":"2023-10-14T01:27:50.544989+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:27:50.544989+0000","last_undegraded":"2023-10-14T01:27:50.544989+0000","last_fullsized":"2023-10-14T01:27:50.544989+0000","mapping_epoch":13,"log_start":"136'313515","ondisk_log_start":"136'313515","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3798409569,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":318630,"num_read_kb":1249326,"num_write":316977,"num_write_kb":1267718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"138'329398","reported_seq":645086,"reported_epoch":138,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:27:48.120117+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:27:48.120117+0000","last_peered":"2023-10-14T01:27:48.120117+0000","last_clean":"2023-10-14T01:27:48.120117+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:27:48.120117+0000","last_undegraded":"2023-10-14T01:27:48.120117+0000","last_fullsized":"2023-10-14T01:27:48.120117+0000","mapping_epoch":13,"log_start":"136'319317","ondisk_log_start":"136'319317","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1711,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 727s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3708424628,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":315938,"num_read_kb":1229591,"num_write":309178,"num_write_kb":1236355,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":351,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:26:58.853338+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:26:58.853338+0000","last_peered":"2023-10-14T01:26:58.853338+0000","last_clean":"2023-10-14T01:26:58.853338+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:26:58.853338+0000","last_undegraded":"2023-10-14T01:26:58.853338+0000","last_fullsized":"2023-10-14T01:26:58.853338+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29030784044,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2457604,"num_read_kb":9597886,"num_write":2422988,"num_write_kb":9689869,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19845758976,"data_stored":29721843800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80466,"ondisk_log_size":80466,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739422,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5541204,"kb_used_data":3785804,"kb_used_omap":486,"kb_used_meta":1754905,"kb_avail":88830636,"statfs":{"total":96636764160,"available":90962571264,"internally_reserved":0,"allocated":3876663296,"data_stored":5804975568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1797023412},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:26:57 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.505,"15min":0.63400000000000001},"min":{"1min":0.22900000000000001,"5min":0.20100000000000001,"15min":0.182},"max":{"1min":2.621,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.245},{"interface":"front","average":{"1min":1.2729999999999999,"5min":0.88600000000000001,"15min":1.024},"min":{"1min":0.19600000000000001,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":10.488,"5min":10.488,"15min":11.481999999999999},"last":0.30099999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:27:01 2023","interfaces":[{"interface":"back","average":{"1min":1.7689999999999999,"5min":1.962,"15min":2.161},"min":{"1min":0.24099999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.4909999999999997,"5min":34.158000000000001,"15min":44.308},"last":4.1059999999999999},{"interface":"front","average":{"1min":0.81799999999999995,"5min":1.8009999999999999,"15min":2.1539999999999999},"min":{"1min":0.28499999999999998,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":4.6100000000000003,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.28499999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739411,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11317288,"kb_used_data":9687964,"kb_used_omap":1271,"kb_used_meta":1628040,"kb_avail":83054552,"statfs":{"total":96636764160,"available":85047861248,"internally_reserved":0,"allocated":9920475136,"data_stored":14858358627,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":1667113256},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,7,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":0.95399999999999996,"15min":0.82899999999999996},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":16.481000000000002,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.20399999999999999},{"interface":"front","average":{"1min":1.1599999999999999,"5min":0.94599999999999995,"15min":0.98799999999999999},"min":{"1min":0.245,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":16.52,"5min":16.52,"15min":16.52},"last":0.25600000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.748,"15min":1},"min":{"1min":0.315,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.6340000000000003,"5min":10.375999999999999,"15min":10.375999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.86399999999999999,"15min":0.81299999999999994},"min":{"1min":0.245,"5min":0.193,"15min":0.193},"max":{"1min":2.6909999999999998,"5min":10.308,"15min":10.308},"last":3.6800000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739424,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7774140,"kb_used_data":5908148,"kb_used_omap":720,"kb_used_meta":1865263,"kb_avail":86597700,"statfs":{"total":96636764160,"available":88676044800,"internally_reserved":0,"allocated":6049943552,"data_stored":9058822826,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":737329,"internal_metadata":1910030287},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":2.024,"5min":1.575,"15min":1.2689999999999999},"min":{"1min":0.20399999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":10.837999999999999,"5min":10.897,"15min":19.274000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":1.0149999999999999,"15min":0.93899999999999995},"min":{"1min":0.214,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":6.0499999999999998,"5min":10.669,"15min":19.376000000000001},"last":0.44}]},{"osd":2,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.73699999999999999,"15min":0.86199999999999999},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":7.1319999999999997,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.27100000000000002},{"interface":"front","average":{"1min":1.222,"5min":0.86799999999999999,"15min":0.85699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":7.6289999999999996,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.48699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6048940032,"data_stored":9057978227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9920876544,"data_stored":14858918956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3875942400,"data_stored":5804946617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:27:53.766 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:53.767 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:53.767 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:27:53.767 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:27:54.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:54.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:54.549 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:54.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:54.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:54.571+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:54.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:54.679+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:27:54.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:54.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:27:54.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:54.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:27:54.732 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:54.727+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:27:54.788 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:54.787+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:27:54.806 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:27:54.803+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:27:56.099 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:27:56.095+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:27:56.100 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:27:56.095+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:27:56.100 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:27:56.095+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T01:27:59.920 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:59.919+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:27:59.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:27:59.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:00.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:00.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:00.013 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:00.011+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:00.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:00.011+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:00.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:00.011+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:00.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:00.015+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:01.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:01.415+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:01.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:01.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:01.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:01.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:01.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:01.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:01.479+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:01.486 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:01.483+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:01.717 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:01.715+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:01.730 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:01.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:03.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:03.523+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:03.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:03.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:03.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:03.543+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:03.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:03.543+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:04.062 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:04.059+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:07.267 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:07.263+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:07.312 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:07.311+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:07.763 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:07.759+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:28:07.787 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:07.783+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:08.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:08.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.587+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:08.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:08.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:08.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:08.660 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.655+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:08.661 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.655+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:08.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:08.659+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:09.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:09.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:09.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:09.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:09.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:09.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:09.063 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:09.059+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:09.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:09.059+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:09.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:09.067+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:09.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:09.119+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:10.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:10.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.075+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:10.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:10.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:10.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:10.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.119+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:10.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.119+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:10.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:10.143+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:13.769 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:28:13.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.846+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:13.903 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.898+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:13.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:13.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:13.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.926+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:13.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.926+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:13.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.926+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:13.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:13.962+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:14.516 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:28:14.517 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:28:14.579 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2795,"stamp":"2023-10-14T01:28:12.946285+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29063982668,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2465545,"num_read_kb":9629179,"num_write":2431126,"num_write_kb":9724293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80659,"ondisk_log_size":80659,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25029612,"kb_used_data":19446600,"kb_used_omap":2478,"kb_used_meta":5580497,"kb_avail":258085908,"statfs":{"total":289910292480,"available":264279969792,"internally_reserved":0,"allocated":19913318400,"data_stored":29788384091,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2537557,"internal_metadata":5714429867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,8,20,15,23,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16187392,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3831,"num_read_kb":15288,"num_write":3963,"num_write_kb":15852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001575"},"pg_stats":[{"pgid":"2.7","version":"139'284866","reported_seq":567054,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:28:08.926394+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:28:08.926394+0000","last_peered":"2023-10-14T01:28:08.926394+0000","last_clean":"2023-10-14T01:28:08.926394+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:28:08.926394+0000","last_undegraded":"2023-10-14T01:28:08.926394+0000","last_fullsized":"2023-10-14T01:28:08.926394+0000","mapping_epoch":13,"log_start":"136'274811","ondisk_log_start":"136'274811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":266,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 94s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3382620343,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":282983,"num_read_kb":1116672,"num_write":283628,"num_write_kb":1134497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"139'299980","reported_seq":592033,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:28:08.896284+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:28:08.896284+0000","last_peered":"2023-10-14T01:28:08.896284+0000","last_clean":"2023-10-14T01:28:08.896284+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:28:08.896284+0000","last_undegraded":"2023-10-14T01:28:08.896284+0000","last_fullsized":"2023-10-14T01:28:08.896284+0000","mapping_epoch":13,"log_start":"136'289908","ondisk_log_start":"136'289908","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":337,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 99s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3410940171,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":293911,"num_read_kb":1137857,"num_write":285012,"num_write_kb":1139647,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"139'306792","reported_seq":608744,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:08.896776+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:28:08.896776+0000","last_peered":"2023-10-14T01:28:08.896776+0000","last_clean":"2023-10-14T01:28:08.896776+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:28:08.896776+0000","last_undegraded":"2023-10-14T01:28:08.896776+0000","last_fullsized":"2023-10-14T01:28:08.896776+0000","mapping_epoch":13,"log_start":"136'296714","ondisk_log_start":"136'296714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3585102379,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":302476,"num_read_kb":1189902,"num_write":299903,"num_write_kb":1199491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"139'324705","reported_seq":637847,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:08.889454+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:28:08.889454+0000","last_peered":"2023-10-14T01:28:08.889454+0000","last_clean":"2023-10-14T01:28:08.889454+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:28:08.889454+0000","last_undegraded":"2023-10-14T01:28:08.889454+0000","last_fullsized":"2023-10-14T01:28:08.889454+0000","mapping_epoch":13,"log_start":"136'314613","ondisk_log_start":"136'314613","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3685314991,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":313618,"num_read_kb":1220883,"num_write":308845,"num_write_kb":1235062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"139'311798","reported_seq":623371,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:11.253289+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:28:11.253289+0000","last_peered":"2023-10-14T01:28:11.253289+0000","last_clean":"2023-10-14T01:28:11.253289+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:28:11.253289+0000","last_undegraded":"2023-10-14T01:28:11.253289+0000","last_fullsized":"2023-10-14T01:28:11.253289+0000","mapping_epoch":13,"log_start":"136'301712","ondisk_log_start":"136'301712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3630281175,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":311400,"num_read_kb":1204979,"num_write":304185,"num_write_kb":1216233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"139'327844","reported_seq":651936,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:10.836153+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:28:10.836153+0000","last_peered":"2023-10-14T01:28:10.836153+0000","last_clean":"2023-10-14T01:28:10.836153+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:28:10.836153+0000","last_undegraded":"2023-10-14T01:28:10.836153+0000","last_fullsized":"2023-10-14T01:28:10.836153+0000","mapping_epoch":13,"log_start":"136'317812","ondisk_log_start":"136'317812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3854193060,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":324430,"num_read_kb":1271740,"num_write":321245,"num_write_kb":1284806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"139'324587","reported_seq":643829,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:10.831109+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:28:10.831109+0000","last_peered":"2023-10-14T01:28:10.831109+0000","last_clean":"2023-10-14T01:28:10.831109+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:28:10.831109+0000","last_undegraded":"2023-10-14T01:28:10.831109+0000","last_fullsized":"2023-10-14T01:28:10.831109+0000","mapping_epoch":13,"log_start":"136'314515","ondisk_log_start":"136'314515","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3802538337,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":319637,"num_read_kb":1253354,"num_write":317986,"num_write_kb":1271754,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"138'330462","reported_seq":647209,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:28:08.864332+0000","last_change":"2023-10-14T01:15:40.807508+0000","last_active":"2023-10-14T01:28:08.864332+0000","last_peered":"2023-10-14T01:28:08.864332+0000","last_clean":"2023-10-14T01:28:08.864332+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:28:08.864332+0000","last_undegraded":"2023-10-14T01:28:08.864332+0000","last_fullsized":"2023-10-14T01:28:08.864332+0000","mapping_epoch":13,"log_start":"136'320417","ondisk_log_start":"136'320417","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1766,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 748s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3712401844,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":316964,"num_read_kb":1233683,"num_write":310153,"num_write_kb":1240255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":352,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:43.055129+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:27:43.055129+0000","last_peered":"2023-10-14T01:27:43.055129+0000","last_clean":"2023-10-14T01:27:43.055129+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:27:43.055129+0000","last_undegraded":"2023-10-14T01:27:43.055129+0000","last_fullsized":"2023-10-14T01:27:43.055129+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29063392300,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2465419,"num_read_kb":9629070,"num_write":2430957,"num_write_kb":9721745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":10447,"num_omap_keys":199,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19910897664,"data_stored":29786982488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80532,"ondisk_log_size":80532,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739426,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5627220,"kb_used_data":3798344,"kb_used_omap":486,"kb_used_meta":1828377,"kb_avail":88744620,"statfs":{"total":96636764160,"available":90874490880,"internally_reserved":0,"allocated":3889504256,"data_stored":5817811855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1872258740},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,6,5,6,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.47299999999999998,"15min":0.60599999999999998},"min":{"1min":0.23799999999999999,"5min":0.20399999999999999,"15min":0.182},"max":{"1min":0.95699999999999996,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.0209999999999999,"15min":1.0529999999999999},"min":{"1min":0.21299999999999999,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":8.2170000000000005,"5min":10.488,"15min":11.481999999999999},"last":0.83599999999999997}]},{"osd":1,"last update":"Sat Oct 14 01:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.696,"15min":2.1200000000000001},"min":{"1min":0.28599999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.343,"5min":34.158000000000001,"15min":44.308},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.38,"5min":1.395,"15min":2.0569999999999999},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.498,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.51700000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739415,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11503084,"kb_used_data":9721060,"kb_used_omap":1271,"kb_used_meta":1780744,"kb_avail":82868756,"statfs":{"total":96636764160,"available":84857606144,"internally_reserved":0,"allocated":9954365440,"data_stored":14892248931,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":1823482152},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,13,8,17,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":0.95399999999999996,"15min":0.82899999999999996},"min":{"1min":0.20899999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":16.481000000000002,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.435},{"interface":"front","average":{"1min":1.1599999999999999,"5min":0.94599999999999995,"15min":0.98799999999999999},"min":{"1min":0.245,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":16.52,"5min":16.52,"15min":16.52},"last":0.46999999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.748,"15min":1},"min":{"1min":0.315,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.6340000000000003,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.505},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.86399999999999999,"15min":0.81299999999999994},"min":{"1min":0.245,"5min":0.193,"15min":0.193},"max":{"1min":2.6909999999999998,"5min":10.308,"15min":10.308},"last":0.27200000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739428,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7899308,"kb_used_data":5927196,"kb_used_omap":720,"kb_used_meta":1971375,"kb_avail":86472532,"statfs":{"total":96636764160,"available":88547872768,"internally_reserved":0,"allocated":6069448704,"data_stored":9078323305,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":737329,"internal_metadata":2018688975},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":2.024,"5min":1.575,"15min":1.2689999999999999},"min":{"1min":0.20399999999999999,"5min":0.19400000000000001,"15min":0.183},"max":{"1min":10.837999999999999,"5min":10.897,"15min":19.274000000000001},"last":0.247},{"interface":"front","average":{"1min":0.78700000000000003,"5min":1.0149999999999999,"15min":0.93899999999999995},"min":{"1min":0.214,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":6.0499999999999998,"5min":10.669,"15min":19.376000000000001},"last":0.41899999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:27:29 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.73699999999999999,"15min":0.86199999999999999},"min":{"1min":0.185,"5min":0.185,"15min":0.185},"max":{"1min":7.1319999999999997,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.29899999999999999},{"interface":"front","average":{"1min":1.222,"5min":0.86799999999999999,"15min":0.85699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":7.6289999999999996,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6068838400,"data_stored":9077876595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9953742848,"data_stored":14891785260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3888316416,"data_stored":5817320633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:28:14.581 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:28:14.581 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:28:14.582 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:28:14.582 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:28:15.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:15.602+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:15.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:15.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:15.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:15.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:15.675 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:15.670+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:16.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:16.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:16.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:16.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:16.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:16.134+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:16.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:16.134+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:16.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:16.166+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:16.919 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:16.914+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:17.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:17.206+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:17.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:17.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:17.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:17.286+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:17.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:17.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:17.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:17.298+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:17.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:17.298+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:17.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:17.358+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:18.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:18.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:18.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:18.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:18.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:18.650+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:20.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:20.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:20.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:20.154+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:20.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:20.190+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:20.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:20.222+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:23.233 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:23.230+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:28:23.241 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:23.238+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:25.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:25.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:25.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:25.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:25.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:25.626+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:25.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:25.626+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:26.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:26.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:26.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:26.526+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:26.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:26.526+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:26.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:26.530+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:29.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.026+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:28:29.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:29.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:29.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.102+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:29.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:29.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:29.130 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.122+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:29.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.122+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:29.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.150+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:29.691 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.682+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:29.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:29.714+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:32.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:32.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:32.969 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:32.966+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:32.982 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:32.978+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:33.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:33.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:33.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:33.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.042+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:33.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.042+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:33.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.050+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:33.419 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.414+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:33.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:33.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:33.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:33.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.506+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:33.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.506+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:33.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:33.506+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:34.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:28:35.451 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:28:35.452 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:28:35.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:35.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:35.483 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:35.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:35.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:35.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:35.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:35.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:35.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:35.482+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:35.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:35.482+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:35.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:35.510+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:35.517 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2806,"stamp":"2023-10-14T01:28:34.953863+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29101809228,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2474931,"num_read_kb":9666603,"num_write":2440357,"num_write_kb":9761217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80556,"ondisk_log_size":80556,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25395816,"kb_used_data":19517392,"kb_used_omap":2478,"kb_used_meta":5875921,"kb_avail":257719704,"statfs":{"total":289910292480,"available":263904976896,"internally_reserved":0,"allocated":19985809408,"data_stored":29860870426,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2537557,"internal_metadata":6016944043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,8,9,10,10,14,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21708800,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5448,"num_read_kb":21736,"num_write":5300,"num_write_kb":21200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003694"},"pg_stats":[{"pgid":"2.7","version":"139'286098","reported_seq":569498,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:28:34.832235+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:28:34.832235+0000","last_peered":"2023-10-14T01:28:34.832235+0000","last_clean":"2023-10-14T01:28:34.832235+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:28:34.832235+0000","last_undegraded":"2023-10-14T01:28:34.832235+0000","last_fullsized":"2023-10-14T01:28:34.832235+0000","mapping_epoch":13,"log_start":"136'276011","ondisk_log_start":"136'276011","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":382,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 120s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3387351223,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":284178,"num_read_kb":1121440,"num_write":284781,"num_write_kb":1139109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"139'301307","reported_seq":594621,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:28:34.843559+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:28:34.843559+0000","last_peered":"2023-10-14T01:28:34.843559+0000","last_clean":"2023-10-14T01:28:34.843559+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:28:34.843559+0000","last_undegraded":"2023-10-14T01:28:34.843559+0000","last_fullsized":"2023-10-14T01:28:34.843559+0000","mapping_epoch":13,"log_start":"136'291208","ondisk_log_start":"136'291208","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":435,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 125s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3415929099,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":295148,"num_read_kb":1142769,"num_write":286228,"num_write_kb":1144511,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"139'308037","reported_seq":611298,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:34.819384+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:28:34.819384+0000","last_peered":"2023-10-14T01:28:34.819384+0000","last_clean":"2023-10-14T01:28:34.819384+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:28:34.819384+0000","last_undegraded":"2023-10-14T01:28:34.819384+0000","last_fullsized":"2023-10-14T01:28:34.819384+0000","mapping_epoch":13,"log_start":"136'298014","ondisk_log_start":"136'298014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3590148651,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":303781,"num_read_kb":1195114,"num_write":301132,"num_write_kb":1204407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"139'326060","reported_seq":640469,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:34.810455+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:28:34.810455+0000","last_peered":"2023-10-14T01:28:34.810455+0000","last_clean":"2023-10-14T01:28:34.810455+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:28:34.810455+0000","last_undegraded":"2023-10-14T01:28:34.810455+0000","last_fullsized":"2023-10-14T01:28:34.810455+0000","mapping_epoch":13,"log_start":"136'316013","ondisk_log_start":"136'316013","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3690664367,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":314866,"num_read_kb":1225863,"num_write":310152,"num_write_kb":1240290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"139'312774","reported_seq":625330,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:31.375872+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:28:31.375872+0000","last_peered":"2023-10-14T01:28:31.375872+0000","last_clean":"2023-10-14T01:28:31.375872+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:28:31.375872+0000","last_undegraded":"2023-10-14T01:28:31.375872+0000","last_fullsized":"2023-10-14T01:28:31.375872+0000","mapping_epoch":13,"log_start":"136'302712","ondisk_log_start":"136'302712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3634270679,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":312363,"num_read_kb":1208823,"num_write":305161,"num_write_kb":1220137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"139'328897","reported_seq":654108,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:30.979974+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:28:30.979974+0000","last_peered":"2023-10-14T01:28:30.979974+0000","last_clean":"2023-10-14T01:28:30.979974+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:28:30.979974+0000","last_undegraded":"2023-10-14T01:28:30.979974+0000","last_fullsized":"2023-10-14T01:28:30.979974+0000","mapping_epoch":13,"log_start":"136'318812","ondisk_log_start":"136'318812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3858485668,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":325543,"num_read_kb":1276172,"num_write":322292,"num_write_kb":1288994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"139'325622","reported_seq":645917,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:30.926965+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:28:30.926965+0000","last_peered":"2023-10-14T01:28:30.926965+0000","last_clean":"2023-10-14T01:28:30.926965+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:28:30.926965+0000","last_undegraded":"2023-10-14T01:28:30.926965+0000","last_fullsized":"2023-10-14T01:28:30.926965+0000","mapping_epoch":13,"log_start":"136'315615","ondisk_log_start":"136'315615","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3806744929,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":320682,"num_read_kb":1257514,"num_write":319015,"num_write_kb":1275870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"139'331836","reported_seq":649891,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:34.759181+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:28:34.759181+0000","last_peered":"2023-10-14T01:28:34.759181+0000","last_clean":"2023-10-14T01:28:34.759181+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:28:34.759181+0000","last_undegraded":"2023-10-14T01:28:34.759181+0000","last_fullsized":"2023-10-14T01:28:34.759181+0000","mapping_epoch":13,"log_start":"136'321817","ondisk_log_start":"136'321817","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3717624244,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":318244,"num_read_kb":1238799,"num_write":311427,"num_write_kb":1245351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":352,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:43.055129+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:27:43.055129+0000","last_peered":"2023-10-14T01:27:43.055129+0000","last_clean":"2023-10-14T01:27:43.055129+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:27:43.055129+0000","last_undegraded":"2023-10-14T01:27:43.055129+0000","last_fullsized":"2023-10-14T01:27:43.055129+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29101218860,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2474805,"num_read_kb":9666494,"num_write":2440188,"num_write_kb":9758669,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19984576512,"data_stored":29860661336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80429,"ondisk_log_size":80429,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739430,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5657472,"kb_used_data":3810552,"kb_used_omap":486,"kb_used_meta":1846425,"kb_avail":88714368,"statfs":{"total":96636764160,"available":90843512832,"internally_reserved":0,"allocated":3902005248,"data_stored":5830312847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1890739892},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.47299999999999998,"15min":0.60599999999999998},"min":{"1min":0.23799999999999999,"5min":0.20399999999999999,"15min":0.182},"max":{"1min":0.95699999999999996,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.0209999999999999,"15min":1.0529999999999999},"min":{"1min":0.21299999999999999,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":8.2170000000000005,"5min":10.488,"15min":11.481999999999999},"last":0.253}]},{"osd":1,"last update":"Sat Oct 14 01:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.696,"15min":2.1200000000000001},"min":{"1min":0.28599999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.343,"5min":34.158000000000001,"15min":44.308},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.38,"5min":1.395,"15min":2.0569999999999999},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.498,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.46100000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739420,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11760212,"kb_used_data":9760396,"kb_used_omap":1271,"kb_used_meta":1998536,"kb_avail":82611628,"statfs":{"total":96636764160,"available":84594307072,"internally_reserved":0,"allocated":9994645504,"data_stored":14932524322,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":2046501160},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,5,10,10,14,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:28:31 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.2,"15min":0.89200000000000002},"min":{"1min":0.20399999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":16.329999999999998,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.44},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.94599999999999995,"15min":0.97999999999999998},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.1499999999999999,"5min":16.52,"15min":16.52},"last":0.40699999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:27:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.748,"15min":1},"min":{"1min":0.315,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":4.6340000000000003,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.86399999999999999,"15min":0.81299999999999994},"min":{"1min":0.245,"5min":0.193,"15min":0.193},"max":{"1min":2.6909999999999998,"5min":10.308,"15min":10.308},"last":2.5859999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739432,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7978132,"kb_used_data":5946444,"kb_used_omap":720,"kb_used_meta":2030959,"kb_avail":86393708,"statfs":{"total":96636764160,"available":88467156992,"internally_reserved":0,"allocated":6089158656,"data_stored":9098033257,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":737329,"internal_metadata":2079702991},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.464,"15min":1.3},"min":{"1min":0.20300000000000001,"5min":0.20200000000000001,"15min":0.183},"max":{"1min":5.5460000000000003,"5min":10.837999999999999,"15min":19.274000000000001},"last":4.9699999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.83499999999999996,"15min":0.90900000000000003},"min":{"1min":0.22700000000000001,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":2.5510000000000002,"5min":6.0499999999999998,"15min":19.376000000000001},"last":0.42899999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.67900000000000005,"15min":0.83899999999999997},"min":{"1min":0.251,"5min":0.185,"15min":0.185},"max":{"1min":2.532,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.86099999999999999,"15min":0.87},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":6.5640000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.39600000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6088617984,"data_stored":9097656179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":9995116544,"data_stored":14933158956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3900841984,"data_stored":5829846201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:28:35.519 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:28:35.519 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:28:35.520 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:28:38.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:38.762+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:38.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:38.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:38.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:38.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:38.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:38.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:38.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:38.790+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:38.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:38.790+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:38.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:38.866+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:39.860 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.858+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:39.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.866+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:39.900 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:39.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:39.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.922+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:39.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.922+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:39.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.922+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:39.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:39.922+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:40.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.074+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:28:40.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.086+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:40.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:40.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:40.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:40.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:40.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.130+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:40.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.130+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:40.134 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:40.130+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:40.350 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:40.342+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:40.356 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:40.354+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:41.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:41.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:41.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.106+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:41.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.106+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:41.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.106+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:41.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:41.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.358+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:41.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:41.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:41.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.378+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:41.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:41.378+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:43.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.050+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:43.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.070+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:43.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:43.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:43.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.110+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:43.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.110+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:43.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:28:43.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:28:43.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:43.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:43.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:43.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:43.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:43.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:43.126+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:44.095 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:44.090+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:28:44.095 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:44.090+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:44.110 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:44.106+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:45.480 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:45.478+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:28:45.486 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:45.482+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:45.486 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:45.482+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:47.140 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:47.138+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:28:47.155 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:47.150+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:47.206 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.202+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:47.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:47.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:47.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:47.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.222+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:47.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.222+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:47.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:47.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:47.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:47.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.346+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:47.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:47.346+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:47.797 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:47.793+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:48.441 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:48.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:48.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:48.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:48.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:48.473+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:48.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:48.473+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:48.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:48.473+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:48.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:48.521+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:50.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:50.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.437+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:50.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.437+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:50.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.437+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:50.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:50.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:50.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.533+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:50.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.533+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:50.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:50.533+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:52.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:52.353+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:52.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:52.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:52.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:52.373+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:52.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:52.373+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:52.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:52.373+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:52.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:52.425+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:54.015 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:54.013+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:28:54.016 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:54.013+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:55.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:55.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:55.108 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:55.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:55.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.105+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:55.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.105+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:55.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.105+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:55.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.149+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:55.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:55.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:55.321 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:55.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:55.521 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:28:56.022 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:56.017+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:56.190 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:28:56.191 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:28:56.262 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2816,"stamp":"2023-10-14T01:28:54.961090+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29129956940,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2481731,"num_read_kb":9693719,"num_write":2447215,"num_write_kb":9788649,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80536,"ondisk_log_size":80536,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24662848,"kb_used_data":19573608,"kb_used_omap":2558,"kb_used_meta":5086657,"kb_avail":258452672,"statfs":{"total":289910292480,"available":264655536128,"internally_reserved":0,"allocated":20043374592,"data_stored":29918435610,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2620321,"internal_metadata":5208736863},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,22,15,20,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16711680,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4043,"num_read_kb":16124,"num_write":4068,"num_write_kb":16272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004090"},"pg_stats":[{"pgid":"2.7","version":"139'286855","reported_seq":570976,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:28:50.251104+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:28:50.251104+0000","last_peered":"2023-10-14T01:28:50.251104+0000","last_clean":"2023-10-14T01:28:50.251104+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:28:50.251104+0000","last_undegraded":"2023-10-14T01:28:50.251104+0000","last_fullsized":"2023-10-14T01:28:50.251104+0000","mapping_epoch":13,"log_start":"136'276811","ondisk_log_start":"136'276811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":412,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 135s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3390365879,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":284891,"num_read_kb":1124292,"num_write":285517,"num_write_kb":1142053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"139'302049","reported_seq":596096,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:28:50.299107+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:28:50.299107+0000","last_peered":"2023-10-14T01:28:50.299107+0000","last_clean":"2023-10-14T01:28:50.299107+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:28:50.299107+0000","last_undegraded":"2023-10-14T01:28:50.299107+0000","last_fullsized":"2023-10-14T01:28:50.299107+0000","mapping_epoch":13,"log_start":"136'292008","ondisk_log_start":"136'292008","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":486,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 140s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3418710283,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":295864,"num_read_kb":1145621,"num_write":286904,"num_write_kb":1147215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"139'308800","reported_seq":612889,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:50.283696+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:28:50.283696+0000","last_peered":"2023-10-14T01:28:50.283696+0000","last_clean":"2023-10-14T01:28:50.283696+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:28:50.283696+0000","last_undegraded":"2023-10-14T01:28:50.283696+0000","last_fullsized":"2023-10-14T01:28:50.283696+0000","mapping_epoch":13,"log_start":"136'298714","ondisk_log_start":"136'298714","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3593241131,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":304604,"num_read_kb":1198402,"num_write":301884,"num_write_kb":1207415,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"139'326897","reported_seq":642083,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:50.294723+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:28:50.294723+0000","last_peered":"2023-10-14T01:28:50.294723+0000","last_clean":"2023-10-14T01:28:50.294723+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:28:50.294723+0000","last_undegraded":"2023-10-14T01:28:50.294723+0000","last_fullsized":"2023-10-14T01:28:50.294723+0000","mapping_epoch":13,"log_start":"137'316813","ondisk_log_start":"137'316813","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3693957551,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":315635,"num_read_kb":1228927,"num_write":310956,"num_write_kb":1243506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"139'313719","reported_seq":627273,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:51.550747+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:28:51.550747+0000","last_peered":"2023-10-14T01:28:51.550747+0000","last_clean":"2023-10-14T01:28:51.550747+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:28:51.550747+0000","last_undegraded":"2023-10-14T01:28:51.550747+0000","last_fullsized":"2023-10-14T01:28:51.550747+0000","mapping_epoch":13,"log_start":"136'303712","ondisk_log_start":"136'303712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3638112727,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":313341,"num_read_kb":1212711,"num_write":306096,"num_write_kb":1223877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"139'330067","reported_seq":656322,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:51.241960+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:28:51.241960+0000","last_peered":"2023-10-14T01:28:51.241960+0000","last_clean":"2023-10-14T01:28:51.241960+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:28:51.241960+0000","last_undegraded":"2023-10-14T01:28:51.241960+0000","last_fullsized":"2023-10-14T01:28:51.241960+0000","mapping_epoch":13,"log_start":"137'320012","ondisk_log_start":"137'320012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3863232932,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":326579,"num_read_kb":1280308,"num_write":323444,"num_write_kb":1293602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"139'326705","reported_seq":648004,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:51.265397+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:28:51.265397+0000","last_peered":"2023-10-14T01:28:51.265397+0000","last_clean":"2023-10-14T01:28:51.265397+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:28:51.265397+0000","last_undegraded":"2023-10-14T01:28:51.265397+0000","last_fullsized":"2023-10-14T01:28:51.265397+0000","mapping_epoch":13,"log_start":"137'316615","ondisk_log_start":"137'316615","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3811098977,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":321678,"num_read_kb":1261478,"num_write":320080,"num_write_kb":1280130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"139'332619","reported_seq":651455,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:50.251302+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:28:50.251302+0000","last_peered":"2023-10-14T01:28:50.251302+0000","last_clean":"2023-10-14T01:28:50.251302+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:28:50.251302+0000","last_undegraded":"2023-10-14T01:28:50.251302+0000","last_fullsized":"2023-10-14T01:28:50.251302+0000","mapping_epoch":13,"log_start":"137'322617","ondisk_log_start":"137'322617","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3720647092,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":319013,"num_read_kb":1241871,"num_write":312165,"num_write_kb":1248303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":352,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:43.055129+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:27:43.055129+0000","last_peered":"2023-10-14T01:27:43.055129+0000","last_clean":"2023-10-14T01:27:43.055129+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:27:43.055129+0000","last_undegraded":"2023-10-14T01:27:43.055129+0000","last_fullsized":"2023-10-14T01:27:43.055129+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29129366572,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2481605,"num_read_kb":9693610,"num_write":2447046,"num_write_kb":9786101,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20042518528,"data_stored":29918603352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":42529,"internal_metadata":0},"log_size":80409,"ondisk_log_size":80409,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739434,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5720080,"kb_used_data":3823176,"kb_used_omap":486,"kb_used_meta":1896409,"kb_avail":88651760,"statfs":{"total":96636764160,"available":90779402240,"internally_reserved":0,"allocated":3914932224,"data_stored":5843239823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":1941923508},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:27:59 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.47299999999999998,"15min":0.60599999999999998},"min":{"1min":0.23799999999999999,"5min":0.20399999999999999,"15min":0.182},"max":{"1min":0.95699999999999996,"5min":4.4569999999999999,"15min":6.1120000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.0209999999999999,"15min":1.0529999999999999},"min":{"1min":0.21299999999999999,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":8.2170000000000005,"5min":10.488,"15min":11.481999999999999},"last":0.27400000000000002}]},{"osd":1,"last update":"Sat Oct 14 01:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.696,"15min":2.1200000000000001},"min":{"1min":0.28599999999999998,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":6.343,"5min":34.158000000000001,"15min":44.308},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.38,"5min":1.395,"15min":2.0569999999999999},"min":{"1min":0.27600000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.498,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.56499999999999995}]}]},{"osd":1,"up_from":8,"seq":34359739423,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11857120,"kb_used_data":9785112,"kb_used_omap":1271,"kb_used_meta":2070728,"kb_avail":82514720,"statfs":{"total":96636764160,"available":84495073280,"internally_reserved":0,"allocated":10019954688,"data_stored":14957833506,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1302232,"internal_metadata":2120425768},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,14,6,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:28:31 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.2,"15min":0.89200000000000002},"min":{"1min":0.20399999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":16.329999999999998,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.254},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.94599999999999995,"15min":0.97999999999999998},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.1499999999999999,"5min":16.52,"15min":16.52},"last":0.32200000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:28:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.73999999999999999,"15min":0.97499999999999998},"min":{"1min":0.28999999999999998,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.2349999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84899999999999998,"15min":0.80000000000000004},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":3.6800000000000002,"5min":10.308,"15min":10.308},"last":0.374}]}]},{"osd":0,"up_from":8,"seq":34359739436,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7085648,"kb_used_data":5965320,"kb_used_omap":800,"kb_used_meta":1119519,"kb_avail":87286192,"statfs":{"total":96636764160,"available":89381060608,"internally_reserved":0,"allocated":6108487680,"data_stored":9117362281,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1146387587},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.464,"15min":1.3},"min":{"1min":0.20300000000000001,"5min":0.20200000000000001,"15min":0.183},"max":{"1min":5.5460000000000003,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.83499999999999996,"15min":0.90900000000000003},"min":{"1min":0.22700000000000001,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":2.5510000000000002,"5min":6.0499999999999998,"15min":19.376000000000001},"last":2.1640000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.67900000000000005,"15min":0.83899999999999997},"min":{"1min":0.251,"5min":0.185,"15min":0.185},"max":{"1min":2.532,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.498},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.86099999999999999,"15min":0.87},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":6.5640000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":4.0979999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6108172288,"data_stored":9117210483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10020323328,"data_stored":14958365740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":18800,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3914022912,"data_stored":5843027129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:28:56.263 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:28:56.264 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:28:56.264 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:28:57.498 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:57.493+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:28:57.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:57.505+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:28:57.510 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:57.505+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:28:57.511 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:28:57.505+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:28:57.878 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:57.873+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:57.918 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:57.913+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:57.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:57.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:57.935 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:57.929+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:57.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:57.929+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:57.936 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:57.929+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:57.938 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:57.933+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:58.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:58.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:28:58.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:58.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:28:58.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.609+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:28:58.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.609+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:28:58.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.609+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:28:58.621 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:58.617+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:28:59.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:59.513+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:28:59.532 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:59.529+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:28:59.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:28:59.565+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:03.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:03.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:03.598 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:03.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:03.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:03.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:03.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:03.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:03.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:03.625+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:03.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:03.625+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:03.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:03.625+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:04.874 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:04.869+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:04.877 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:04.873+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:29:04.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:04.885+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:05.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:05.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:05.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:05.265+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:05.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:05.269+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:05.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:05.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:05.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:05.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:05.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:05.281+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:05.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:05.285+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:06.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.113+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:06.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.181+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:06.190 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:06.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:06.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:06.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.185+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:06.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.185+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:06.581 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:06.577+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:06.585 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:06.581+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:06.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.673+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:06.678 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.673+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:06.713 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.709+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:06.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:06.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.713+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:06.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.713+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:06.720 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.713+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:06.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:06.737+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:07.371 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:07.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:07.403 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.393+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:07.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:07.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.401+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:07.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.401+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:07.407 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.401+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:07.409 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:07.405+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:08.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:08.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:08.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:08.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:08.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:08.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:08.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:08.065+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:08.411 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:08.405+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:08.430 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:08.425+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:29:08.431 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:08.425+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:10.018 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:10.013+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:10.019 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:10.013+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:10.594 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:10.589+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:10.616 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:10.613+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:29:10.622 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:10.617+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:10.788 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:10.777+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:29:10.809 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:10.781+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:12.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:12.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:12.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:12.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:12.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:12.385+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:12.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:12.385+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:12.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:12.385+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:12.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:12.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:12.870 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:12.865+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:13.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:13.893+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:13.917 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:13.909+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:13.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:13.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:13.927 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:13.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:13.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:13.921+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:13.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:13.921+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:14.966 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:14.961+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:15.075 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:15.069+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:15.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.477+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:15.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.481+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:15.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.493+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:15.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:15.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:15.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:15.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.497+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:15.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:15.509+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:16.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.089+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:29:16.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.089+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:29:16.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.109+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:16.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:16.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:16.141 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:16.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:16.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:16.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.221+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:16.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.221+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:16.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:16.229+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:16.265 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:29:16.986 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:29:16.987 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:29:17.024 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2827,"stamp":"2023-10-14T01:29:16.964628+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29172092492,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2492133,"num_read_kb":9735219,"num_write":2457490,"num_write_kb":9829749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80692,"ondisk_log_size":80692,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24895536,"kb_used_data":19656792,"kb_used_omap":2046,"kb_used_meta":5236673,"kb_avail":258219984,"statfs":{"total":289910292480,"available":264417263616,"internally_reserved":0,"allocated":20128555008,"data_stored":30003616026,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2095936,"internal_metadata":5362353344},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,6,5,9,16,16,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25358336,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6196,"num_read_kb":24728,"num_write":6193,"num_write_kb":24772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001913"},"pg_stats":[{"pgid":"2.7","version":"139'288151","reported_seq":573490,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:29:16.082648+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:29:16.082648+0000","last_peered":"2023-10-14T01:29:16.082648+0000","last_clean":"2023-10-14T01:29:16.082648+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:29:16.082648+0000","last_undegraded":"2023-10-14T01:29:16.082648+0000","last_fullsized":"2023-10-14T01:29:16.082648+0000","mapping_epoch":13,"log_start":"137'278111","ondisk_log_start":"137'278111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":501,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 161s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3395436727,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":286097,"num_read_kb":1129104,"num_write":286751,"num_write_kb":1146989,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"139'303408","reported_seq":598716,"reported_epoch":139,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:29:16.020898+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:29:16.020898+0000","last_peered":"2023-10-14T01:29:16.020898+0000","last_clean":"2023-10-14T01:29:16.020898+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:29:16.020898+0000","last_undegraded":"2023-10-14T01:29:16.020898+0000","last_fullsized":"2023-10-14T01:29:16.020898+0000","mapping_epoch":13,"log_start":"137'293308","ondisk_log_start":"137'293308","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":563,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 166s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3423854859,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":297100,"num_read_kb":1150553,"num_write":288155,"num_write_kb":1152219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"139'310107","reported_seq":615498,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:29:16.038559+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:29:16.038559+0000","last_peered":"2023-10-14T01:29:16.038559+0000","last_clean":"2023-10-14T01:29:16.038559+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:29:16.038559+0000","last_undegraded":"2023-10-14T01:29:16.038559+0000","last_fullsized":"2023-10-14T01:29:16.038559+0000","mapping_epoch":13,"log_start":"137'300014","ondisk_log_start":"137'300014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3598541355,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":305902,"num_read_kb":1203574,"num_write":303173,"num_write_kb":1212571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"139'328290","reported_seq":644904,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:29:16.035101+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:29:16.035101+0000","last_peered":"2023-10-14T01:29:16.035101+0000","last_clean":"2023-10-14T01:29:16.035101+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:29:16.035101+0000","last_undegraded":"2023-10-14T01:29:16.035101+0000","last_fullsized":"2023-10-14T01:29:16.035101+0000","mapping_epoch":13,"log_start":"137'318213","ondisk_log_start":"137'318213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3699442095,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":317055,"num_read_kb":1234591,"num_write":312295,"num_write_kb":1248862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"139'315007","reported_seq":629849,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:29:16.795898+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:29:16.795898+0000","last_peered":"2023-10-14T01:29:16.795898+0000","last_clean":"2023-10-14T01:29:16.795898+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:29:16.795898+0000","last_undegraded":"2023-10-14T01:29:16.795898+0000","last_fullsized":"2023-10-14T01:29:16.795898+0000","mapping_epoch":13,"log_start":"137'304912","ondisk_log_start":"137'304912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3643331031,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":314605,"num_read_kb":1217759,"num_write":307369,"num_write_kb":1228969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"139'331457","reported_seq":659163,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:29:16.419480+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:29:16.419480+0000","last_peered":"2023-10-14T01:29:16.419480+0000","last_clean":"2023-10-14T01:29:16.419480+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:29:16.419480+0000","last_undegraded":"2023-10-14T01:29:16.419480+0000","last_fullsized":"2023-10-14T01:29:16.419480+0000","mapping_epoch":13,"log_start":"137'321412","ondisk_log_start":"137'321412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3868799396,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":328022,"num_read_kb":1286060,"num_write":324806,"num_write_kb":1299050,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"139'327999","reported_seq":650554,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:29:16.436724+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:29:16.436724+0000","last_peered":"2023-10-14T01:29:16.436724+0000","last_clean":"2023-10-14T01:29:16.436724+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:29:16.436724+0000","last_undegraded":"2023-10-14T01:29:16.436724+0000","last_fullsized":"2023-10-14T01:29:16.436724+0000","mapping_epoch":13,"log_start":"137'317915","ondisk_log_start":"137'317915","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3816292705,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":322928,"num_read_kb":1266470,"num_write":321346,"num_write_kb":1285194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"139'333948","reported_seq":654088,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:29:16.030195+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:29:16.030195+0000","last_peered":"2023-10-14T01:29:16.030195+0000","last_clean":"2023-10-14T01:29:16.030195+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:29:16.030195+0000","last_undegraded":"2023-10-14T01:29:16.030195+0000","last_fullsized":"2023-10-14T01:29:16.030195+0000","mapping_epoch":13,"log_start":"137'323917","ondisk_log_start":"137'323917","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3725803956,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":320298,"num_read_kb":1246999,"num_write":313426,"num_write_kb":1253347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":352,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-10-14T01:27:43.055129+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:27:43.055129+0000","last_peered":"2023-10-14T01:27:43.055129+0000","last_clean":"2023-10-14T01:27:43.055129+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:27:43.055129+0000","last_undegraded":"2023-10-14T01:27:43.055129+0000","last_fullsized":"2023-10-14T01:27:43.055129+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29171502124,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2492007,"num_read_kb":9735110,"num_write":2457321,"num_write_kb":9827201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20126457856,"data_stored":30002542680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80565,"ondisk_log_size":80565,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739439,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5845664,"kb_used_data":3838552,"kb_used_omap":486,"kb_used_meta":2006617,"kb_avail":88526176,"statfs":{"total":96636764160,"available":90650804224,"internally_reserved":0,"allocated":3930677248,"data_stored":5858984847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":2054776500},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.54700000000000004,"15min":0.60499999999999998},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.182},"max":{"1min":3.5800000000000001,"5min":4.4569999999999999,"15min":5.2629999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.19,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":5.9749999999999996,"5min":10.488,"15min":11.481999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Sat Oct 14 01:29:01 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.571,"15min":2.0859999999999999},"min":{"1min":0.29699999999999999,"5min":0.21199999999999999,"15min":0.20999999999999999},"max":{"1min":11.154,"5min":34.158000000000001,"15min":44.308},"last":0.316},{"interface":"front","average":{"1min":0.628,"5min":1.109,"15min":1.9690000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.1680000000000001,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.34200000000000003}]}]},{"osd":1,"up_from":8,"seq":34359739428,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11856608,"kb_used_data":9827224,"kb_used_omap":759,"kb_used_meta":2028616,"kb_avail":82515232,"statfs":{"total":96636764160,"available":84495597568,"internally_reserved":0,"allocated":10063077376,"data_stored":15000956194,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":777847,"internal_metadata":2077303177},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,5,4,5,8,10,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:28:31 2023","interfaces":[{"interface":"back","average":{"1min":1.575,"5min":1.2,"15min":0.89200000000000002},"min":{"1min":0.20399999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":16.329999999999998,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.94599999999999995,"15min":0.97999999999999998},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.1499999999999999,"5min":16.52,"15min":16.52},"last":0.27100000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:28:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.73999999999999999,"15min":0.97499999999999998},"min":{"1min":0.28999999999999998,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.2349999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84899999999999998,"15min":0.80000000000000004},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":3.6800000000000002,"5min":10.308,"15min":10.308},"last":0.222}]}]},{"osd":0,"up_from":8,"seq":34359739441,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7193264,"kb_used_data":5991016,"kb_used_omap":800,"kb_used_meta":1201439,"kb_avail":87178576,"statfs":{"total":96636764160,"available":89270861824,"internally_reserved":0,"allocated":6134800384,"data_stored":9143674985,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1230273667},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:28:30 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.464,"15min":1.3},"min":{"1min":0.20300000000000001,"5min":0.20200000000000001,"15min":0.183},"max":{"1min":5.5460000000000003,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.83499999999999996,"15min":0.90900000000000003},"min":{"1min":0.22700000000000001,"5min":0.20999999999999999,"15min":0.20799999999999999},"max":{"1min":2.5510000000000002,"5min":6.0499999999999998,"15min":19.376000000000001},"last":2.073}]},{"osd":2,"last update":"Sat Oct 14 01:28:30 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.67900000000000005,"15min":0.83899999999999997},"min":{"1min":0.251,"5min":0.185,"15min":0.185},"max":{"1min":2.532,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.37},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.86099999999999999,"15min":0.87},"min":{"1min":0.26000000000000001,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":6.5640000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.32300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6133862400,"data_stored":9142900595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10062770176,"data_stored":15000812588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3929825280,"data_stored":5858829497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:29:17.026 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:29:17.026 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:29:17.027 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:29:17.039 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:17.029+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:17.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:17.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:17.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:17.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:17.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:17.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:17.085 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:17.077+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:17.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:17.077+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:17.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:17.089+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:18.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:18.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:18.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.277+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:18.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.277+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:18.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.277+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:18.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.293+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:18.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.361+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:29:18.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.377+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:18.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.377+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:18.386 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.381+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:18.393 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:18.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:18.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.389+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:18.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.389+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:18.395 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.389+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:18.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:18.401+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:19.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:19.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:19.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:19.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:19.562 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.557+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:19.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.569+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:19.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.597+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:19.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:19.807 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.801+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:19.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.817+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:19.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:19.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.821+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:19.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.821+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:19.826 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:19.821+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:21.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:21.145 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:21.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:21.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:21.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.165+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:21.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.165+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:21.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.289+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:21.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:21.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:21.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:21.349 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:21.333+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:22.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:22.308 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:22.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:22.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:22.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:22.338 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:22.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:22.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:22.333+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:25.930 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:25.924+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:25.971 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:25.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:26.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:26.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.024+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:26.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.024+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:26.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.032+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:26.044 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.036+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:26.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:26.059 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:26.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.112+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:26.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:26.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.120+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:26.128 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.120+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:26.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.120+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:29:26.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.120+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:29:26.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.140+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:26.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:26.162 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.152+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:26.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.160+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:26.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:26.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:26.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.176+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:26.182 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.176+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:26.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.176+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:26.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.180+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:29:26.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.180+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:29:26.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.220+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:26.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:26.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:26.292 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:26.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:26.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.288+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:26.293 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.288+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:26.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:26.300+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:26.825 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:26.820+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:29:27.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.292+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:29:27.304 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.292+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:27.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:27.357 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:27.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:27.378 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:27.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.376+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:27.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.376+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:27.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.376+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:27.406 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:27.396+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:29.894 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:29.888+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:29.931 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:29.928+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:30.145 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:30.140+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:29:30.176 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:30.172+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:30.663 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:30.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:30.683 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:30.676+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:30.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:30.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:30.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:30.688+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:30.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:30.688+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:30.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:30.688+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:30.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:30.708+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:31.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:31.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:31.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:31.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.700+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:31.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.700+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:31.707 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.700+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:31.775 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.768+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:31.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:31.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:31.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.788+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:31.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.788+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:31.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.792+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:31.803 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:31.800+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:32.450 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:32.444+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:32.468 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:32.464+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:33.033 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.020+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:33.050 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.044+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:33.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:33.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.052+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:33.057 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.052+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:33.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.052+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:33.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.092+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:33.465 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:29:33.460+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:29:33.465 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:29:33.460+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:29:33.975 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:33.956+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:34.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:34.860+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:34.891 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:34.884+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:34.933 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:34.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:34.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:34.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:34.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:34.928+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:34.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:34.928+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:34.934 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:34.928+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:35.183 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:35.176+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:35.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:35.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:35.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:35.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:35.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:35.236+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:35.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:35.236+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:35.554 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:35.540+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:35.578 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:35.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:37.028 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:29:37.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.752+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:29:37.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.780+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:37.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.804+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:37.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.820+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:37.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:37.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:37.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.832+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:37.838 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.832+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:37.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:37.840+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:37.970 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:29:37.971 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:29:38.052 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2838,"stamp":"2023-10-14T01:29:36.967862+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29205397068,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2500203,"num_read_kb":9767443,"num_write":2465611,"num_write_kb":9862233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80479,"ondisk_log_size":80479,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25220164,"kb_used_data":19720420,"kb_used_omap":2046,"kb_used_meta":5497665,"kb_avail":257895356,"statfs":{"total":289910292480,"available":264084844544,"internally_reserved":0,"allocated":20193710080,"data_stored":30068756950,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2095936,"internal_metadata":5629609152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,6,2,10,9,9,21],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25255936,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6083,"num_read_kb":24276,"num_write":6152,"num_write_kb":24608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001917"},"pg_stats":[{"pgid":"2.7","version":"140'289161","reported_seq":575436,"reported_epoch":140,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:29:36.753876+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:29:36.753876+0000","last_peered":"2023-10-14T01:29:36.753876+0000","last_clean":"2023-10-14T01:29:36.753876+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:29:36.753876+0000","last_undegraded":"2023-10-14T01:29:36.753876+0000","last_fullsized":"2023-10-14T01:29:36.753876+0000","mapping_epoch":13,"log_start":"137'279111","ondisk_log_start":"137'279111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":572,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 182s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3399377079,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":287020,"num_read_kb":1132796,"num_write":287713,"num_write_kb":1150837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"140'304479","reported_seq":600776,"reported_epoch":140,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:29:36.744773+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:29:36.744773+0000","last_peered":"2023-10-14T01:29:36.744773+0000","last_clean":"2023-10-14T01:29:36.744773+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:29:36.744773+0000","last_undegraded":"2023-10-14T01:29:36.744773+0000","last_fullsized":"2023-10-14T01:29:36.744773+0000","mapping_epoch":13,"log_start":"137'294408","ondisk_log_start":"137'294408","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":634,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 187s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3427758347,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":298069,"num_read_kb":1154425,"num_write":289110,"num_write_kb":1156039,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"140'311156","reported_seq":617532,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:36.709902+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:29:36.709902+0000","last_peered":"2023-10-14T01:29:36.709902+0000","last_clean":"2023-10-14T01:29:36.709902+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:29:36.709902+0000","last_undegraded":"2023-10-14T01:29:36.709902+0000","last_fullsized":"2023-10-14T01:29:36.709902+0000","mapping_epoch":13,"log_start":"137'301114","ondisk_log_start":"137'301114","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3602739755,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":306883,"num_read_kb":1207482,"num_write":304201,"num_write_kb":1216683,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"140'329461","reported_seq":647176,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:36.669124+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:29:36.669124+0000","last_peered":"2023-10-14T01:29:36.669124+0000","last_clean":"2023-10-14T01:29:36.669124+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:29:36.669124+0000","last_undegraded":"2023-10-14T01:29:36.669124+0000","last_fullsized":"2023-10-14T01:29:36.669124+0000","mapping_epoch":13,"log_start":"137'319413","ondisk_log_start":"137'319413","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3703992751,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":318138,"num_read_kb":1238915,"num_write":313404,"num_write_kb":1253298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"140'315985","reported_seq":631832,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:36.948299+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:29:36.948299+0000","last_peered":"2023-10-14T01:29:36.948299+0000","last_clean":"2023-10-14T01:29:36.948299+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:29:36.948299+0000","last_undegraded":"2023-10-14T01:29:36.948299+0000","last_fullsized":"2023-10-14T01:29:36.948299+0000","mapping_epoch":13,"log_start":"137'305912","ondisk_log_start":"137'305912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3647304151,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":315597,"num_read_kb":1221719,"num_write":308332,"num_write_kb":1232821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"140'332539","reported_seq":661314,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:36.628633+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:29:36.628633+0000","last_peered":"2023-10-14T01:29:36.628633+0000","last_clean":"2023-10-14T01:29:36.628633+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:29:36.628633+0000","last_undegraded":"2023-10-14T01:29:36.628633+0000","last_fullsized":"2023-10-14T01:29:36.628633+0000","mapping_epoch":13,"log_start":"137'322512","ondisk_log_start":"137'322512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3873186212,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":329084,"num_read_kb":1290296,"num_write":325880,"num_write_kb":1303346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"140'329025","reported_seq":652623,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:36.627942+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:29:36.627942+0000","last_peered":"2023-10-14T01:29:36.627942+0000","last_clean":"2023-10-14T01:29:36.627942+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:29:36.627942+0000","last_undegraded":"2023-10-14T01:29:36.627942+0000","last_fullsized":"2023-10-14T01:29:36.627942+0000","mapping_epoch":13,"log_start":"137'319015","ondisk_log_start":"137'319015","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3820470625,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":323961,"num_read_kb":1270598,"num_write":322364,"num_write_kb":1289266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"140'335048","reported_seq":656230,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:36.723063+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:29:36.723063+0000","last_peered":"2023-10-14T01:29:36.723063+0000","last_clean":"2023-10-14T01:29:36.723063+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:29:36.723063+0000","last_undegraded":"2023-10-14T01:29:36.723063+0000","last_fullsized":"2023-10-14T01:29:36.723063+0000","mapping_epoch":13,"log_start":"137'325017","ondisk_log_start":"137'325017","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3729977780,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":321325,"num_read_kb":1251103,"num_write":314438,"num_write_kb":1257395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":353,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-10-14T01:28:08.843653+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:28:08.843653+0000","last_peered":"2023-10-14T01:28:08.843653+0000","last_clean":"2023-10-14T01:28:08.843653+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:28:08.843653+0000","last_undegraded":"2023-10-14T01:28:08.843653+0000","last_fullsized":"2023-10-14T01:28:08.843653+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29204806700,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2500077,"num_read_kb":9767334,"num_write":2465442,"num_write_kb":9859685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20192759808,"data_stored":30068844632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80352,"ondisk_log_size":80352,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739443,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5875888,"kb_used_data":3850728,"kb_used_omap":486,"kb_used_meta":2024665,"kb_avail":88495952,"statfs":{"total":96636764160,"available":90619854848,"internally_reserved":0,"allocated":3943145472,"data_stored":5871448355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":497996,"internal_metadata":2073257652},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.54700000000000004,"15min":0.60499999999999998},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.182},"max":{"1min":3.5800000000000001,"5min":4.4569999999999999,"15min":5.2629999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.19,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":5.9749999999999996,"5min":10.488,"15min":11.481999999999999},"last":10.692}]},{"osd":1,"last update":"Sat Oct 14 01:29:01 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.571,"15min":2.0859999999999999},"min":{"1min":0.29699999999999999,"5min":0.21199999999999999,"15min":0.20999999999999999},"max":{"1min":11.154,"5min":34.158000000000001,"15min":44.308},"last":5.3550000000000004},{"interface":"front","average":{"1min":0.628,"5min":1.109,"15min":1.9690000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.1680000000000001,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.42999999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739432,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12037100,"kb_used_data":9859164,"kb_used_omap":759,"kb_used_meta":2177160,"kb_avail":82334740,"statfs":{"total":96636764160,"available":84310773760,"internally_reserved":0,"allocated":10095783936,"data_stored":15033658038,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":777847,"internal_metadata":2229412233},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,3,1,7,4,5,16],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.262,"15min":0.92000000000000004},"min":{"1min":0.22500000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":5.2759999999999998,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.314},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.84699999999999998,"15min":0.97099999999999997},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.1059999999999999,"5min":16.52,"15min":16.52},"last":0.37}]},{"osd":2,"last update":"Sat Oct 14 01:28:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.73999999999999999,"15min":0.97499999999999998},"min":{"1min":0.28999999999999998,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":2.2349999999999999,"5min":10.375999999999999,"15min":10.375999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.84899999999999998,"15min":0.80000000000000004},"min":{"1min":0.22600000000000001,"5min":0.193,"15min":0.193},"max":{"1min":3.6800000000000002,"5min":10.308,"15min":10.308},"last":6.8369999999999997}]}]},{"osd":0,"up_from":8,"seq":34359739445,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7307176,"kb_used_data":6010528,"kb_used_omap":800,"kb_used_meta":1295839,"kb_avail":87064664,"statfs":{"total":96636764160,"available":89154215936,"internally_reserved":0,"allocated":6154780672,"data_stored":9163650557,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1326939267},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,4,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.4590000000000001,"15min":1.3069999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":5.5300000000000002,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.434},{"interface":"front","average":{"1min":2.605,"5min":1.181,"15min":1.05},"min":{"1min":0.215,"5min":0.214,"15min":0.20799999999999999},"max":{"1min":13.875,"5min":13.875,"15min":19.376000000000001},"last":2.1739999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68700000000000006,"15min":0.78000000000000003},"min":{"1min":0.215,"5min":0.185,"15min":0.185},"max":{"1min":5.5949999999999998,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92800000000000005,"15min":0.90400000000000003},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":5.617,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.59399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6154031104,"data_stored":9163069299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10096320512,"data_stored":15034362924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3942408192,"data_stored":5871412409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:29:38.053 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:29:38.054 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:29:38.054 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:29:39.113 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:39.108+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:39.140 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:39.136+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:43.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:43.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:43.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:43.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:43.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:43.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:43.507 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:43.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:43.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:43.500+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:43.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:43.500+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:43.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:43.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:44.246 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:44.240+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:44.283 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:44.276+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:46.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:46.168+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:46.699 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:46.688+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:29:46.714 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:46.708+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:29:46.720 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:46.712+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:48.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:48.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.368+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:48.377 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.372+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:48.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:48.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:48.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:48.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.380+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:48.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.408+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:48.882 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.872+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:29:48.895 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:48.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:48.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:48.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:48.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:48.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:48.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:48.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:48.944+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:50.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.552+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:29:50.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.580+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:50.605 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:50.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:50.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:50.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.604+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:50.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.604+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:50.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:50.612+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:52.559 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:52.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:52.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:52.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:52.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:52.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:52.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:52.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:52.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:52.564+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:52.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:52.564+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:52.574 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:52.568+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:52.779 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:52.772+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:53.114 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:29:53.112+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:29:53.599 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:53.588+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:53.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:53.636+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:54.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.036+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:54.058 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:54.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.064+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:54.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:54.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:54.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.076+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:54.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.076+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:54.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:54.084+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:56.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:56.984+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:29:57.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:57.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:29:57.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:57.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:57.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:57.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.008+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:57.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.008+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:57.012 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:57.012+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:29:58.056 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:29:58.759 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:29:58.759 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:29:58.797 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2848,"stamp":"2023-10-14T01:29:56.976793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29232193100,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2506585,"num_read_kb":9792919,"num_write":2472160,"num_write_kb":9888429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80593,"ondisk_log_size":80593,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23953828,"kb_used_data":19773452,"kb_used_omap":2095,"kb_used_meta":4178256,"kb_avail":259161692,"statfs":{"total":289910292480,"available":265381572608,"internally_reserved":0,"allocated":20248014848,"data_stored":30123061718,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2145616,"internal_metadata":4278534832},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,12,11,14,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18681856,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4324,"num_read_kb":17260,"num_write":4563,"num_write_kb":18252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007607"},"pg_stats":[{"pgid":"2.7","version":"140'289894","reported_seq":576839,"reported_epoch":140,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:29:52.114617+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:29:52.114617+0000","last_peered":"2023-10-14T01:29:52.114617+0000","last_clean":"2023-10-14T01:29:52.114617+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:29:52.114617+0000","last_undegraded":"2023-10-14T01:29:52.114617+0000","last_fullsized":"2023-10-14T01:29:52.114617+0000","mapping_epoch":13,"log_start":"137'279811","ondisk_log_start":"137'279811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":633,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 197s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3402199223,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":287682,"num_read_kb":1135440,"num_write":288406,"num_write_kb":1153609,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"140'305236","reported_seq":602177,"reported_epoch":140,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:29:52.117720+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:29:52.117720+0000","last_peered":"2023-10-14T01:29:52.117720+0000","last_clean":"2023-10-14T01:29:52.117720+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:29:52.117720+0000","last_undegraded":"2023-10-14T01:29:52.117720+0000","last_fullsized":"2023-10-14T01:29:52.117720+0000","mapping_epoch":13,"log_start":"137'295208","ondisk_log_start":"137'295208","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":661,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 202s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3430703371,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":298699,"num_read_kb":1156945,"num_write":289831,"num_write_kb":1158923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"140'311893","reported_seq":619027,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:52.105763+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:29:52.105763+0000","last_peered":"2023-10-14T01:29:52.105763+0000","last_clean":"2023-10-14T01:29:52.105763+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:29:52.105763+0000","last_undegraded":"2023-10-14T01:29:52.105763+0000","last_fullsized":"2023-10-14T01:29:52.105763+0000","mapping_epoch":13,"log_start":"137'301814","ondisk_log_start":"137'301814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3605717547,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":307642,"num_read_kb":1210506,"num_write":304931,"num_write_kb":1219603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"140'330256","reported_seq":648719,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:52.072056+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:29:52.072056+0000","last_peered":"2023-10-14T01:29:52.072056+0000","last_clean":"2023-10-14T01:29:52.072056+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:29:52.072056+0000","last_undegraded":"2023-10-14T01:29:52.072056+0000","last_fullsized":"2023-10-14T01:29:52.072056+0000","mapping_epoch":13,"log_start":"137'320213","ondisk_log_start":"137'320213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3707167151,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":318877,"num_read_kb":1241867,"num_write":314180,"num_write_kb":1256402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"140'316712","reported_seq":633318,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:52.036349+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:29:52.036349+0000","last_peered":"2023-10-14T01:29:52.036349+0000","last_clean":"2023-10-14T01:29:52.036349+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:29:52.036349+0000","last_undegraded":"2023-10-14T01:29:52.036349+0000","last_fullsized":"2023-10-14T01:29:52.036349+0000","mapping_epoch":13,"log_start":"137'306612","ondisk_log_start":"137'306612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3650253271,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":316338,"num_read_kb":1224667,"num_write":309049,"num_write_kb":1235689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"140'333643","reported_seq":663426,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:56.818657+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:29:56.818657+0000","last_peered":"2023-10-14T01:29:56.818657+0000","last_clean":"2023-10-14T01:29:56.818657+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:29:56.818657+0000","last_undegraded":"2023-10-14T01:29:56.818657+0000","last_fullsized":"2023-10-14T01:29:56.818657+0000","mapping_epoch":13,"log_start":"137'323612","ondisk_log_start":"137'323612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3877671332,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":330084,"num_read_kb":1294288,"num_write":326970,"num_write_kb":1307706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"140'330115","reported_seq":654771,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:56.786909+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:29:56.786909+0000","last_peered":"2023-10-14T01:29:56.786909+0000","last_clean":"2023-10-14T01:29:56.786909+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:29:56.786909+0000","last_undegraded":"2023-10-14T01:29:56.786909+0000","last_fullsized":"2023-10-14T01:29:56.786909+0000","mapping_epoch":13,"log_start":"137'320015","ondisk_log_start":"137'320015","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3824857441,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":325010,"num_read_kb":1274790,"num_write":323440,"num_write_kb":1293570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"140'335819","reported_seq":657818,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:52.097074+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:29:52.097074+0000","last_peered":"2023-10-14T01:29:52.097074+0000","last_clean":"2023-10-14T01:29:52.097074+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:29:52.097074+0000","last_undegraded":"2023-10-14T01:29:52.097074+0000","last_fullsized":"2023-10-14T01:29:52.097074+0000","mapping_epoch":13,"log_start":"137'325817","ondisk_log_start":"137'325817","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3733033396,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":322127,"num_read_kb":1254307,"num_write":315184,"num_write_kb":1260379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":354,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:18.879038+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:29:18.879038+0000","last_peered":"2023-10-14T01:29:18.879038+0000","last_clean":"2023-10-14T01:29:18.879038+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:29:18.879038+0000","last_undegraded":"2023-10-14T01:29:18.879038+0000","last_fullsized":"2023-10-14T01:29:18.879038+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29231602732,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2506459,"num_read_kb":9792810,"num_write":2471991,"num_write_kb":9885881,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20245086208,"data_stored":30121171032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80466,"ondisk_log_size":80466,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739447,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4592432,"kb_used_data":3863848,"kb_used_omap":534,"kb_used_meta":728041,"kb_avail":89779408,"statfs":{"total":96636764160,"available":91934113792,"internally_reserved":0,"allocated":3956580352,"data_stored":5884883235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":745514148},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:29:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.54700000000000004,"15min":0.60499999999999998},"min":{"1min":0.21099999999999999,"5min":0.20399999999999999,"15min":0.182},"max":{"1min":3.5800000000000001,"5min":4.4569999999999999,"15min":5.2629999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.94199999999999995,"5min":1.0049999999999999,"15min":1.04},"min":{"1min":0.19,"5min":0.16400000000000001,"15min":0.16400000000000001},"max":{"1min":5.9749999999999996,"5min":10.488,"15min":11.481999999999999},"last":0.193}]},{"osd":1,"last update":"Sat Oct 14 01:29:01 2023","interfaces":[{"interface":"back","average":{"1min":1.375,"5min":1.571,"15min":2.0859999999999999},"min":{"1min":0.29699999999999999,"5min":0.21199999999999999,"15min":0.20999999999999999},"max":{"1min":11.154,"5min":34.158000000000001,"15min":44.308},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.628,"5min":1.109,"15min":1.9690000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.1680000000000001,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.36599999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739435,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12007944,"kb_used_data":9884736,"kb_used_omap":759,"kb_used_meta":2122440,"kb_avail":82363896,"statfs":{"total":96636764160,"available":84340629504,"internally_reserved":0,"allocated":10121969664,"data_stored":15059843766,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":777847,"internal_metadata":2173378953},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,10,4,6,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.262,"15min":0.92000000000000004},"min":{"1min":0.22500000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":5.2759999999999998,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.84699999999999998,"15min":0.97099999999999997},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.1059999999999999,"5min":16.52,"15min":16.52},"last":0.59199999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.73099999999999998,"15min":0.97499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.3159999999999998,"5min":5.3159999999999998,"15min":10.375999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":0.96499999999999997,"15min":0.877},"min":{"1min":0.222,"5min":0.193,"15min":0.193},"max":{"1min":6.8369999999999997,"5min":9.4309999999999992,"15min":10.308},"last":0.46400000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739448,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7353452,"kb_used_data":6024868,"kb_used_omap":800,"kb_used_meta":1327775,"kb_avail":87018388,"statfs":{"total":96636764160,"available":89106829312,"internally_reserved":0,"allocated":6169464832,"data_stored":9178334717,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1359641731},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.4590000000000001,"15min":1.3069999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":5.5300000000000002,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.25800000000000001},{"interface":"front","average":{"1min":2.605,"5min":1.181,"15min":1.05},"min":{"1min":0.215,"5min":0.214,"15min":0.20799999999999999},"max":{"1min":13.875,"5min":13.875,"15min":19.376000000000001},"last":0.20699999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68700000000000006,"15min":0.78000000000000003},"min":{"1min":0.215,"5min":0.185,"15min":0.185},"max":{"1min":5.5949999999999998,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92800000000000005,"15min":0.90400000000000003},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":5.617,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.443}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6168682496,"data_stored":9177720691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10121011200,"data_stored":15059053612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3955392512,"data_stored":5884396729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:29:58.799 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:29:58.799 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:29:58.799 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:29:59.316 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:59.312+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:29:59.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:59.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:29:59.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:59.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:29:59.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:59.340+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:29:59.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:59.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:29:59.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:59.340+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:29:59.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:29:59.340+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:01.244 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:01.244+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:01.265 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:01.264+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:02.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.963+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:02.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.963+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:02.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:02.976 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:02.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:02.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.983+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:02.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.983+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:02.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.983+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:02.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:02.987+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:03.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:03.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:03.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:03.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:03.094 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:03.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:03.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:03.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:03.135+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:03.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:03.139+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:03.595 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:03.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:03.595 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:03.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:04.357 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:04.355+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:05.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:05.747+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:05.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:05.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:05.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:05.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:05.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:05.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:05.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:05.819+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:05.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:05.819+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:05.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:05.823+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:06.642 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:06.639+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:06.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:06.643+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:06.647 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:06.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:06.747 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:06.739+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:06.748 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:06.739+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:07.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:07.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:07.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:07.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:07.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:07.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.295+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:07.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.295+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:07.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.295+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:07.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.699+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:30:07.700 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.699+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:30:07.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.763+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:07.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.783+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:07.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.795+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:07.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:07.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:07.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:07.806 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.803+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:07.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.815+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:07.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.823+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:30:07.829 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.823+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:30:07.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:07.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:07.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:07.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.919+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:07.923 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.919+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:07.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.919+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:07.939 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:07.935+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:08.656 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:08.655+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:09.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.063+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:09.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:09.082 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:09.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:09.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.099+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:09.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.099+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:09.121 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.119+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:09.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.431+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:09.433 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:09.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.443+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:09.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:09.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:09.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:09.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.487+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:09.490 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.487+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:09.491 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.487+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:09.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.951+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:09.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:09.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:09.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:09.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.987+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:09.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.987+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:09.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:09.987+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:10.163 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:10.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:10.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:10.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:10.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:10.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.215+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:10.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.223+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:10.243 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:10.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:10.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:10.431 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:10.427+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:11.896 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:11.895+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:11.950 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:11.947+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:11.990 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:11.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:11.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:11.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:11.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:11.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:11.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:11.987+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:11.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:11.987+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:12.014 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.011+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:12.226 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.219+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:12.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:12.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:12.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.291+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:12.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.291+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:12.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.291+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:12.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:12.291+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:14.489 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:14.487+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:14.495 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:14.491+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:14.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:14.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:14.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:14.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:17.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:17.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.259+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:17.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:17.390 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:17.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.343+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:17.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.351+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:17.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.383+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:17.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:17.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.767+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:17.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:17.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.815+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:17.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:17.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:17.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:17.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.819+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:17.823 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.819+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:17.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:17.819+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:18.801 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:30:19.743 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:30:19.744 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:30:19.777 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2859,"stamp":"2023-10-14T01:30:18.987004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29271895628,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2516055,"num_read_kb":9830703,"num_write":2481845,"num_write_kb":9927169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80475,"ondisk_log_size":80475,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23012484,"kb_used_data":19850732,"kb_used_omap":2096,"kb_used_meta":3159631,"kb_avail":260103036,"statfs":{"total":289910292480,"available":266345508864,"internally_reserved":0,"allocated":20327149568,"data_stored":30202196438,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":3235462177},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,1,9,0,18,11,10,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22949888,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5489,"num_read_kb":21908,"num_write":5591,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001985"},"pg_stats":[{"pgid":"2.7","version":"140'291170","reported_seq":579303,"reported_epoch":140,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:30:17.978996+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:30:17.978996+0000","last_peered":"2023-10-14T01:30:17.978996+0000","last_clean":"2023-10-14T01:30:17.978996+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:30:17.978996+0000","last_undegraded":"2023-10-14T01:30:17.978996+0000","last_fullsized":"2023-10-14T01:30:17.978996+0000","mapping_epoch":13,"log_start":"137'281111","ondisk_log_start":"137'281111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":732,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 223s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3407155383,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":288858,"num_read_kb":1140128,"num_write":289617,"num_write_kb":1158453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"140'306578","reported_seq":604759,"reported_epoch":140,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:30:17.965887+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:30:17.965887+0000","last_peered":"2023-10-14T01:30:17.965887+0000","last_clean":"2023-10-14T01:30:17.965887+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:30:17.965887+0000","last_undegraded":"2023-10-14T01:30:17.965887+0000","last_fullsized":"2023-10-14T01:30:17.965887+0000","mapping_epoch":13,"log_start":"137'296510","ondisk_log_start":"137'296510","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":739,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 228s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3435725067,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":299909,"num_read_kb":1161781,"num_write":291052,"num_write_kb":1163807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"140'313235","reported_seq":621629,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:30:17.952939+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:30:17.952939+0000","last_peered":"2023-10-14T01:30:17.952939+0000","last_clean":"2023-10-14T01:30:17.952939+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:30:17.952939+0000","last_undegraded":"2023-10-14T01:30:17.952939+0000","last_fullsized":"2023-10-14T01:30:17.952939+0000","mapping_epoch":13,"log_start":"137'303214","ondisk_log_start":"137'303214","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3611116075,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":308891,"num_read_kb":1215490,"num_write":306251,"num_write_kb":1224883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"140'331627","reported_seq":651375,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:30:17.922559+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:30:17.922559+0000","last_peered":"2023-10-14T01:30:17.922559+0000","last_clean":"2023-10-14T01:30:17.922559+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:30:17.922559+0000","last_undegraded":"2023-10-14T01:30:17.922559+0000","last_fullsized":"2023-10-14T01:30:17.922559+0000","mapping_epoch":13,"log_start":"138'321613","ondisk_log_start":"138'321613","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3712516527,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":320147,"num_read_kb":1246939,"num_write":315487,"num_write_kb":1261630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"140'317990","reported_seq":635841,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:30:17.312516+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:30:17.312516+0000","last_peered":"2023-10-14T01:30:17.312516+0000","last_clean":"2023-10-14T01:30:17.312516+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:30:17.312516+0000","last_undegraded":"2023-10-14T01:30:17.312516+0000","last_fullsized":"2023-10-14T01:30:17.312516+0000","mapping_epoch":13,"log_start":"137'307912","ondisk_log_start":"137'307912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3655434711,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":317565,"num_read_kb":1229571,"num_write":310317,"num_write_kb":1240761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"140'334720","reported_seq":665499,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:30:16.961676+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:30:16.961676+0000","last_peered":"2023-10-14T01:30:16.961676+0000","last_clean":"2023-10-14T01:30:16.961676+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:30:16.961676+0000","last_undegraded":"2023-10-14T01:30:16.961676+0000","last_fullsized":"2023-10-14T01:30:16.961676+0000","mapping_epoch":13,"log_start":"138'324712","ondisk_log_start":"138'324712","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3882029476,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":331074,"num_read_kb":1298220,"num_write":328033,"num_write_kb":1311958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"140'331140","reported_seq":656817,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:30:16.959120+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:30:16.959120+0000","last_peered":"2023-10-14T01:30:16.959120+0000","last_clean":"2023-10-14T01:30:16.959120+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:30:16.959120+0000","last_undegraded":"2023-10-14T01:30:16.959120+0000","last_fullsized":"2023-10-14T01:30:16.959120+0000","mapping_epoch":13,"log_start":"138'321115","ondisk_log_start":"138'321115","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3829035361,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":326026,"num_read_kb":1278834,"num_write":324450,"num_write_kb":1297610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"140'337192","reported_seq":660543,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:30:17.932691+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:30:17.932691+0000","last_peered":"2023-10-14T01:30:17.932691+0000","last_clean":"2023-10-14T01:30:17.932691+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:30:17.932691+0000","last_undegraded":"2023-10-14T01:30:17.932691+0000","last_fullsized":"2023-10-14T01:30:17.932691+0000","mapping_epoch":13,"log_start":"138'327117","ondisk_log_start":"138'327117","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3738292660,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":323459,"num_read_kb":1259631,"num_write":316469,"num_write_kb":1265519,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":354,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:18.879038+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:29:18.879038+0000","last_peered":"2023-10-14T01:29:18.879038+0000","last_clean":"2023-10-14T01:29:18.879038+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:29:18.879038+0000","last_undegraded":"2023-10-14T01:29:18.879038+0000","last_fullsized":"2023-10-14T01:29:18.879038+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29271305260,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2515929,"num_read_kb":9830594,"num_write":2481676,"num_write_kb":9924621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20325494784,"data_stored":30201579608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80348,"ondisk_log_size":80348,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739451,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4689228,"kb_used_data":3876036,"kb_used_omap":534,"kb_used_meta":812649,"kb_avail":89682612,"statfs":{"total":96636764160,"available":91834994688,"internally_reserved":0,"allocated":3969060864,"data_stored":5897363747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":832152740},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.245,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":2.3940000000000001,"5min":4.4569999999999999,"15min":5.2629999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.099,"15min":1.0629999999999999},"min":{"1min":0.193,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":10.692,"5min":10.692,"15min":11.481999999999999},"last":0.64200000000000002}]},{"osd":1,"last update":"Sat Oct 14 01:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.619,"15min":1.825},"min":{"1min":0.27900000000000003,"5min":0.24099999999999999,"15min":0.20999999999999999},"max":{"1min":5.3550000000000004,"5min":34.158000000000001,"15min":44.308},"last":1.599},{"interface":"front","average":{"1min":0.371,"5min":0.98599999999999999,"15min":1.6539999999999999},"min":{"1min":0.26100000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45200000000000001,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.71799999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739440,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10838196,"kb_used_data":9925228,"kb_used_omap":761,"kb_used_meta":912198,"kb_avail":83533644,"statfs":{"total":96636764160,"available":85538451456,"internally_reserved":0,"allocated":10163433472,"data_stored":15101307574,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":934091514},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,8,0,11,6,8,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:29:31 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":1.262,"15min":0.92000000000000004},"min":{"1min":0.22500000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":5.2759999999999998,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.84699999999999998,"15min":0.97099999999999997},"min":{"1min":0.22800000000000001,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":2.1059999999999999,"5min":16.52,"15min":16.52},"last":0.433}]},{"osd":2,"last update":"Sat Oct 14 01:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.73099999999999998,"15min":0.97499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.3159999999999998,"5min":5.3159999999999998,"15min":10.375999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":1.5469999999999999,"5min":0.96499999999999997,"15min":0.877},"min":{"1min":0.222,"5min":0.193,"15min":0.193},"max":{"1min":6.8369999999999997,"5min":9.4309999999999992,"15min":10.308},"last":0.33000000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739453,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7485060,"kb_used_data":6049468,"kb_used_omap":800,"kb_used_meta":1434783,"kb_avail":86886780,"statfs":{"total":96636764160,"available":88972062720,"internally_reserved":0,"allocated":6194655232,"data_stored":9203525117,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1469217923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":1.4590000000000001,"15min":1.3069999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.183},"max":{"1min":5.5300000000000002,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.251},{"interface":"front","average":{"1min":2.605,"5min":1.181,"15min":1.05},"min":{"1min":0.215,"5min":0.214,"15min":0.20799999999999999},"max":{"1min":13.875,"5min":13.875,"15min":19.376000000000001},"last":0.20899999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:29:32 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68700000000000006,"15min":0.78000000000000003},"min":{"1min":0.215,"5min":0.185,"15min":0.185},"max":{"1min":5.5949999999999998,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.92800000000000005,"15min":0.90400000000000003},"min":{"1min":0.23699999999999999,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":5.617,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.23599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6194151424,"data_stored":9203189619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10163400704,"data_stored":15101443116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3967942656,"data_stored":5896946873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:30:19.779 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:30:19.779 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:30:19.780 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:30:21.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.023+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:21.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:21.104 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:21.115 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:21.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:21.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.111+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:21.116 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.111+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:21.117 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.111+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:21.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.135+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:30:21.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.135+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:30:21.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.143+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:21.180 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:21.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:21.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:21.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.183+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:21.189 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:21.183+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:22.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.107+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:30:22.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:22.192 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:22.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:22.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.191+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:22.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.191+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:22.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.191+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:22.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:22.191+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:24.546 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:24.543+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:24.566 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:24.563+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:25.453 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:25.451+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:25.752 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:25.751+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:25.753 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:25.751+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:26.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:26.107 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:26.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:26.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:26.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.147+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:26.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.147+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:26.354 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:26.351+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:26.606 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:26.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:26.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:26.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:26.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.635+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:26.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.635+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:26.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.635+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:26.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:26.659+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:27.330 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:27.327+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:27.347 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:27.343+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:28.974 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:28.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:29.002 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:28.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:29.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:28.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:29.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:28.991+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:29.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.547+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:29.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.547+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:29.592 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:29.593 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.591+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:29.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:29.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:29.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.607+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:29.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.607+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:29.617 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:29.611+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:30.472 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:30.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:30.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:30.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:30.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:30.475+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:30.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:30.515+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:35.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:35.447+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:36.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.255+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:36.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.307+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:36.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:36.379 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:36.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.375+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:36.380 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.375+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:36.381 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.375+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:36.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:36.391+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:37.089 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:37.087+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:30:37.101 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:37.099+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:37.163 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:37.163+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:37.201 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:37.195+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:37.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:37.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:37.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:37.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:37.244 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:37.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:37.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:37.239+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:37.245 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:37.239+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:37.247 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:37.243+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:39.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:39.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:39.019 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:39.015+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:39.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:39.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:39.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:39.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:39.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:39.067+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:39.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:39.067+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:39.095 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:39.091+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:39.781 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:30:40.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:40.354+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:40.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:40.386+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:40.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:40.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:40.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:40.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:40.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:40.418+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:40.422 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:40.418+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:40.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:40.454+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:40.618 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:30:40.619 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:30:40.690 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2870,"stamp":"2023-10-14T01:30:38.991407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29305990732,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2524146,"num_read_kb":9862983,"num_write":2490163,"num_write_kb":9960441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80579,"ondisk_log_size":80579,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23364624,"kb_used_data":19918200,"kb_used_omap":2096,"kb_used_meta":3444303,"kb_avail":259750896,"statfs":{"total":289910292480,"available":265984917504,"internally_reserved":0,"allocated":20396236800,"data_stored":30271269393,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":3526966305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,3,18,9,14,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17166336,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4025,"num_read_kb":16056,"num_write":4183,"num_write_kb":16732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001668"},"pg_stats":[{"pgid":"2.7","version":"141'292178","reported_seq":581230,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:30:38.812703+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:30:38.812703+0000","last_peered":"2023-10-14T01:30:38.812703+0000","last_clean":"2023-10-14T01:30:38.812703+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:30:38.812703+0000","last_undegraded":"2023-10-14T01:30:38.812703+0000","last_fullsized":"2023-10-14T01:30:38.812703+0000","mapping_epoch":13,"log_start":"138'282111","ondisk_log_start":"138'282111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":825,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 244s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3411009719,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":289767,"num_read_kb":1143760,"num_write":290560,"num_write_kb":1162225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"141'307629","reported_seq":606812,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:30:38.789695+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:30:38.789695+0000","last_peered":"2023-10-14T01:30:38.789695+0000","last_clean":"2023-10-14T01:30:38.789695+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:30:38.789695+0000","last_undegraded":"2023-10-14T01:30:38.789695+0000","last_fullsized":"2023-10-14T01:30:38.789695+0000","mapping_epoch":13,"log_start":"138'297610","ondisk_log_start":"138'297610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":816,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 249s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3439677707,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":300886,"num_read_kb":1165681,"num_write":292014,"num_write_kb":1167655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"141'314303","reported_seq":623715,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:38.714160+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:30:38.714160+0000","last_peered":"2023-10-14T01:30:38.714160+0000","last_clean":"2023-10-14T01:30:38.714160+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:30:38.714160+0000","last_undegraded":"2023-10-14T01:30:38.714160+0000","last_fullsized":"2023-10-14T01:30:38.714160+0000","mapping_epoch":13,"log_start":"138'304214","ondisk_log_start":"138'304214","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3615466027,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":309905,"num_read_kb":1219542,"num_write":307308,"num_write_kb":1229111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"141'332727","reported_seq":653544,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:38.715239+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:30:38.715239+0000","last_peered":"2023-10-14T01:30:38.715239+0000","last_clean":"2023-10-14T01:30:38.715239+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:30:38.715239+0000","last_undegraded":"2023-10-14T01:30:38.715239+0000","last_fullsized":"2023-10-14T01:30:38.715239+0000","mapping_epoch":13,"log_start":"138'322713","ondisk_log_start":"138'322713","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3716878767,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":321200,"num_read_kb":1251131,"num_write":316553,"num_write_kb":1265894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"141'319059","reported_seq":637941,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:37.641342+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:30:37.641342+0000","last_peered":"2023-10-14T01:30:37.641342+0000","last_clean":"2023-10-14T01:30:37.641342+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:30:37.641342+0000","last_undegraded":"2023-10-14T01:30:37.641342+0000","last_fullsized":"2023-10-14T01:30:37.641342+0000","mapping_epoch":13,"log_start":"138'309012","ondisk_log_start":"138'309012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3659747799,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":318566,"num_read_kb":1233543,"num_write":311371,"num_write_kb":1244977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"141'335859","reported_seq":667707,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:37.125096+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:30:37.125096+0000","last_peered":"2023-10-14T01:30:37.125096+0000","last_clean":"2023-10-14T01:30:37.125096+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:30:37.125096+0000","last_undegraded":"2023-10-14T01:30:37.125096+0000","last_fullsized":"2023-10-14T01:30:37.125096+0000","mapping_epoch":13,"log_start":"138'325812","ondisk_log_start":"138'325812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3886641572,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":332135,"num_read_kb":1302456,"num_write":329158,"num_write_kb":1316458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"141'332193","reported_seq":658931,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:37.163209+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:30:37.163209+0000","last_peered":"2023-10-14T01:30:37.163209+0000","last_clean":"2023-10-14T01:30:37.163209+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:30:37.163209+0000","last_undegraded":"2023-10-14T01:30:37.163209+0000","last_fullsized":"2023-10-14T01:30:37.163209+0000","mapping_epoch":13,"log_start":"138'322115","ondisk_log_start":"138'322115","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3833303393,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":327078,"num_read_kb":1283034,"num_write":325489,"num_write_kb":1301766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"141'338308","reported_seq":662696,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:38.732659+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:30:38.732659+0000","last_peered":"2023-10-14T01:30:38.732659+0000","last_clean":"2023-10-14T01:30:38.732659+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:30:38.732659+0000","last_undegraded":"2023-10-14T01:30:38.732659+0000","last_fullsized":"2023-10-14T01:30:38.732659+0000","mapping_epoch":13,"log_start":"138'328217","ondisk_log_start":"138'328217","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3742675380,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":324483,"num_read_kb":1263727,"num_write":317541,"num_write_kb":1269807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":355,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:46.977403+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:29:46.977403+0000","last_peered":"2023-10-14T01:29:46.977403+0000","last_clean":"2023-10-14T01:29:46.977403+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:29:46.977403+0000","last_undegraded":"2023-10-14T01:29:46.977403+0000","last_fullsized":"2023-10-14T01:29:46.977403+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29305400364,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2524020,"num_read_kb":9862874,"num_write":2489994,"num_write_kb":9957893,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20393381888,"data_stored":30269466712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80452,"ondisk_log_size":80452,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739455,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4733852,"kb_used_data":3888724,"kb_used_omap":534,"kb_used_meta":844585,"kb_avail":89637988,"statfs":{"total":96636764160,"available":91789299712,"internally_reserved":0,"allocated":3982053376,"data_stored":5910351500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":864855204},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,2,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.245,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":2.3940000000000001,"5min":4.4569999999999999,"15min":5.2629999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.099,"15min":1.0629999999999999},"min":{"1min":0.193,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":10.692,"5min":10.692,"15min":11.481999999999999},"last":0.44400000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.619,"15min":1.825},"min":{"1min":0.27900000000000003,"5min":0.24099999999999999,"15min":0.20999999999999999},"max":{"1min":5.3550000000000004,"5min":34.158000000000001,"15min":44.308},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.98599999999999999,"15min":1.6539999999999999},"min":{"1min":0.26100000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45200000000000001,"5min":34.192999999999998,"15min":44.350000000000001},"last":0.59299999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739444,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11026696,"kb_used_data":9959680,"kb_used_omap":761,"kb_used_meta":1066246,"kb_avail":83345144,"statfs":{"total":96636764160,"available":85345427456,"internally_reserved":0,"allocated":10198712320,"data_stored":15136581663,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":1091836666},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,12,2,4,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":1.1499999999999999,"15min":0.91900000000000004},"min":{"1min":0.23100000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":1.0449999999999999,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.216},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.97499999999999998},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.482,"5min":16.52,"15min":16.52},"last":0.84799999999999998}]},{"osd":2,"last update":"Sat Oct 14 01:29:37 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":0.73099999999999998,"15min":0.97499999999999998},"min":{"1min":0.26100000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":5.3159999999999998,"5min":5.3159999999999998,"15min":10.375999999999999},"last":0.25600000000000001},{"interface":"front","average":{"1min":1.5469999999999999,"5min":0.96499999999999997,"15min":0.877},"min":{"1min":0.222,"5min":0.193,"15min":0.193},"max":{"1min":6.8369999999999997,"5min":9.4309999999999992,"15min":10.308},"last":0.316}]}]},{"osd":0,"up_from":8,"seq":34359739457,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7604076,"kb_used_data":6069796,"kb_used_omap":800,"kb_used_meta":1533471,"kb_avail":86767764,"statfs":{"total":96636764160,"available":88850190336,"internally_reserved":0,"allocated":6215471104,"data_stored":9224336230,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1570274435},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.4490000000000001,"15min":1.3029999999999999},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.183},"max":{"1min":5.3760000000000003,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":2.3730000000000002,"5min":1.5329999999999999,"15min":1.1499999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":9.9450000000000003,"5min":13.875,"15min":19.376000000000001},"last":0.81100000000000005}]},{"osd":2,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.77600000000000002,"15min":0.73899999999999999},"min":{"1min":0.252,"5min":0.185,"15min":0.185},"max":{"1min":5.4580000000000002,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.496},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.94499999999999995,"15min":0.86299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":4.4630000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.66200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6214598656,"data_stored":9223636851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10197794816,"data_stored":15135837228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3980988416,"data_stored":5909992633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:30:40.692 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:30:40.692 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:30:40.693 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:30:41.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:41.490+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:41.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:41.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:41.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:41.534+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:41.540 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:41.534+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:41.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:41.534+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:41.675 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:30:41.723 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:30:42.414 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:30:42.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:42.446+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:42.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:42.462+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:42.480 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:42.470+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:42.499 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:30:43.614 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:43.624 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.618+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:43.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:43.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.626+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:43.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.626+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:43.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.634+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:43.659 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.654+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:43.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.962+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:30:43.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.962+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:30:43.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.962+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:43.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:43.986+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:44.009 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.002+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:44.010 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.006+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:44.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:44.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:44.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.022+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:44.030 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.022+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:44.041 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.038+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:44.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.038+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:30:44.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.038+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:30:44.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.046+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:44.049 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:44.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.090+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:44.119 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.114+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:44.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:44.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:44.152 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:44.150+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:45.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.738+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:30:45.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.738+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:45.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.758+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:45.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.770+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:45.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:45.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.786+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:45.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.786+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:45.791 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.786+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:45.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:45.798+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:47.832 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:47.830+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:48.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:48.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:48.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.518+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:48.528 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.518+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:48.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.522+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:48.555 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.546+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:48.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:48.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.546+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:48.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.546+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:48.558 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:48.554+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:30:48.561 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:48.558+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:30:48.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:48.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:48.846+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:49.569 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:49.562+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:49.573 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:49.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:49.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:49.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:49.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:49.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:49.575 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:49.570+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:49.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:49.570+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:50.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:50.222+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:50.260 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:50.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:50.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:50.258+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:50.261 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:50.258+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:50.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:50.294+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:51.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:51.422+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:51.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:51.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:51.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:51.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:51.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:51.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:51.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:51.454+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:51.457 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:51.454+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:57.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:57.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:57.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:57.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:57.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:57.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:30:57.177 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:57.174+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:58.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:58.650+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:30:58.658 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:58.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:30:58.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:58.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:30:58.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:58.678+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:30:58.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:58.678+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:30:58.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:58.678+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:30:58.711 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:30:58.706+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:30:59.559 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:30:59.550+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:00.767 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:00.762+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:00.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:00.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:00.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:00.778+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:00.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:00.778+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:00.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:00.778+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:00.792 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:00.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:01.193 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.190+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:31:01.196 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:01.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:01.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:01.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:01.256 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:01.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:01.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.250+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:01.269 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.266+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:01.390 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:01.386+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:01.398 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:01.394+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:01.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:01.620 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.618+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:01.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:01.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:01.802+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:02.501 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:31:03.286 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:31:03.287 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:31:03.374 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2882,"stamp":"2023-10-14T01:31:02.995248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29342629452,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2532825,"num_read_kb":9897591,"num_write":2499087,"num_write_kb":9996137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80423,"ondisk_log_size":80423,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":23778212,"kb_used_data":19991052,"kb_used_omap":2096,"kb_used_meta":3785039,"kb_avail":259337308,"statfs":{"total":289910292480,"available":265561403392,"internally_reserved":0,"allocated":20470837248,"data_stored":30345869841,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":3875879969},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,8,13,11,12,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20058112,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4751,"num_read_kb":18948,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001964"},"pg_stats":[{"pgid":"2.7","version":"141'293181","reported_seq":583156,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:30:59.275113+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:30:59.275113+0000","last_peered":"2023-10-14T01:30:59.275113+0000","last_clean":"2023-10-14T01:30:59.275113+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:30:59.275113+0000","last_undegraded":"2023-10-14T01:30:59.275113+0000","last_fullsized":"2023-10-14T01:30:59.275113+0000","mapping_epoch":13,"log_start":"138'283111","ondisk_log_start":"138'283111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":913,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 264s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3414872247,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":290679,"num_read_kb":1147400,"num_write":291503,"num_write_kb":1165997,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"141'308667","reported_seq":608804,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:30:59.257755+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:30:59.257755+0000","last_peered":"2023-10-14T01:30:59.257755+0000","last_clean":"2023-10-14T01:30:59.257755+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:30:59.257755+0000","last_undegraded":"2023-10-14T01:30:59.257755+0000","last_fullsized":"2023-10-14T01:30:59.257755+0000","mapping_epoch":13,"log_start":"138'298610","ondisk_log_start":"138'298610","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":863,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 269s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3443605771,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":301819,"num_read_kb":1169397,"num_write":292974,"num_write_kb":1171495,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"141'315371","reported_seq":625821,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:59.243689+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:30:59.243689+0000","last_peered":"2023-10-14T01:30:59.243689+0000","last_clean":"2023-10-14T01:30:59.243689+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:30:59.243689+0000","last_undegraded":"2023-10-14T01:30:59.243689+0000","last_fullsized":"2023-10-14T01:30:59.243689+0000","mapping_epoch":13,"log_start":"138'305314","ondisk_log_start":"138'305314","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3619836459,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":310938,"num_read_kb":1223658,"num_write":308361,"num_write_kb":1233323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"141'333818","reported_seq":655661,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:59.235209+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:30:59.235209+0000","last_peered":"2023-10-14T01:30:59.235209+0000","last_clean":"2023-10-14T01:30:59.235209+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:30:59.235209+0000","last_undegraded":"2023-10-14T01:30:59.235209+0000","last_fullsized":"2023-10-14T01:30:59.235209+0000","mapping_epoch":13,"log_start":"138'323813","ondisk_log_start":"138'323813","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3721163183,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":322220,"num_read_kb":1255199,"num_write":317599,"num_write_kb":1270078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"141'320318","reported_seq":640432,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:02.923320+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:31:02.923320+0000","last_peered":"2023-10-14T01:31:02.923320+0000","last_clean":"2023-10-14T01:31:02.923320+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:31:02.923320+0000","last_undegraded":"2023-10-14T01:31:02.923320+0000","last_fullsized":"2023-10-14T01:31:02.923320+0000","mapping_epoch":13,"log_start":"138'310312","ondisk_log_start":"138'310312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3664830935,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":319773,"num_read_kb":1238359,"num_write":312605,"num_write_kb":1249913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"141'337213","reported_seq":670354,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:02.338778+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:31:02.338778+0000","last_peered":"2023-10-14T01:31:02.338778+0000","last_clean":"2023-10-14T01:31:02.338778+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:31:02.338778+0000","last_undegraded":"2023-10-14T01:31:02.338778+0000","last_fullsized":"2023-10-14T01:31:02.338778+0000","mapping_epoch":13,"log_start":"138'327212","ondisk_log_start":"138'327212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3892113828,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":333416,"num_read_kb":1307560,"num_write":330494,"num_write_kb":1321802,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"141'333472","reported_seq":661484,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:02.324837+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:31:02.324837+0000","last_peered":"2023-10-14T01:31:02.324837+0000","last_clean":"2023-10-14T01:31:02.324837+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:31:02.324837+0000","last_undegraded":"2023-10-14T01:31:02.324837+0000","last_fullsized":"2023-10-14T01:31:02.324837+0000","mapping_epoch":13,"log_start":"138'323415","ondisk_log_start":"138'323415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3838464353,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":328340,"num_read_kb":1288074,"num_write":326750,"num_write_kb":1306810,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"141'339460","reported_seq":664900,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:30:59.193712+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:30:59.193712+0000","last_peered":"2023-10-14T01:30:59.193712+0000","last_clean":"2023-10-14T01:30:59.193712+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:30:59.193712+0000","last_undegraded":"2023-10-14T01:30:59.193712+0000","last_fullsized":"2023-10-14T01:30:59.193712+0000","mapping_epoch":13,"log_start":"138'329417","ondisk_log_start":"138'329417","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3747152308,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":325514,"num_read_kb":1267835,"num_write":318632,"num_write_kb":1274171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":355,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:46.977403+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:29:46.977403+0000","last_peered":"2023-10-14T01:29:46.977403+0000","last_clean":"2023-10-14T01:29:46.977403+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:29:46.977403+0000","last_undegraded":"2023-10-14T01:29:46.977403+0000","last_fullsized":"2023-10-14T01:29:46.977403+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29342039084,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2532699,"num_read_kb":9897482,"num_write":2498918,"num_write_kb":9993589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20468584448,"data_stored":30344669272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80296,"ondisk_log_size":80296,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739460,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4821920,"kb_used_data":3904664,"kb_used_omap":534,"kb_used_meta":916713,"kb_avail":89549920,"statfs":{"total":96636764160,"available":91699118080,"internally_reserved":0,"allocated":3998375936,"data_stored":5926674060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":938714276},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:30:02 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.57599999999999996,"15min":0.59699999999999998},"min":{"1min":0.245,"5min":0.21099999999999999,"15min":0.182},"max":{"1min":2.3940000000000001,"5min":4.4569999999999999,"15min":5.2629999999999999},"last":5.3140000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":1.099,"15min":1.0629999999999999},"min":{"1min":0.193,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":10.692,"5min":10.692,"15min":11.481999999999999},"last":3.2650000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:30:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.619,"15min":1.825},"min":{"1min":0.27900000000000003,"5min":0.24099999999999999,"15min":0.20999999999999999},"max":{"1min":5.3550000000000004,"5min":34.158000000000001,"15min":44.308},"last":5.2050000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.98599999999999999,"15min":1.6539999999999999},"min":{"1min":0.26100000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.45200000000000001,"5min":34.192999999999998,"15min":44.350000000000001},"last":5.266}]}]},{"osd":1,"up_from":8,"seq":34359739448,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11200912,"kb_used_data":9991944,"kb_used_omap":761,"kb_used_meta":1208198,"kb_avail":83170928,"statfs":{"total":96636764160,"available":85167030272,"internally_reserved":0,"allocated":10231750656,"data_stored":15169619999,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":1237195514},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,11,6,6,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":1.1499999999999999,"15min":0.91900000000000004},"min":{"1min":0.23100000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":1.0449999999999999,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.97499999999999998},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.482,"5min":16.52,"15min":16.52},"last":0.40400000000000003}]},{"osd":2,"last update":"Sat Oct 14 01:30:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.80800000000000005,"15min":0.98599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.21299999999999999},"max":{"1min":6.5789999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.224,"5min":0.193,"15min":0.193},"max":{"1min":2.206,"5min":6.8369999999999997,"15min":10.308},"last":0.216}]}]},{"osd":0,"up_from":8,"seq":34359739462,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7755380,"kb_used_data":6094444,"kb_used_omap":800,"kb_used_meta":1660127,"kb_avail":86616460,"statfs":{"total":96636764160,"available":88695255040,"internally_reserved":0,"allocated":6240710656,"data_stored":9249575782,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1699970179},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.4490000000000001,"15min":1.3029999999999999},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.183},"max":{"1min":5.3760000000000003,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":2.3730000000000002,"5min":1.5329999999999999,"15min":1.1499999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":9.9450000000000003,"5min":13.875,"15min":19.376000000000001},"last":5.5300000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.77600000000000002,"15min":0.73899999999999999},"min":{"1min":0.252,"5min":0.185,"15min":0.185},"max":{"1min":5.4580000000000002,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.94499999999999995,"15min":0.86299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":4.4630000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.253}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6239911936,"data_stored":9248950131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10231468032,"data_stored":15169510444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3997204480,"data_stored":5926208697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:31:03.375 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:31:03.376 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:31:03.376 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:31:04.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:04.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:04.650 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:04.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:04.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:04.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:04.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:04.646+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:04.652 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:04.646+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:04.674 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:04.670+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:05.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:05.434+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:05.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:05.450+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:05.459 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:05.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:05.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:05.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:05.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:05.462+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:06.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:06.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:06.848 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:06.834+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:06.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:06.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:06.850 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:06.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:06.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:06.838+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:06.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:06.838+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:06.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:06.854+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:07.765 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.762+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:31:07.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.790+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:07.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:07.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:07.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.826+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:07.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.826+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:07.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.826+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:07.901 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:07.898+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:10.273 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:10.274 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.270+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:10.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:10.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:10.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.326+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:10.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.326+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:10.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.326+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:10.387 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:10.374+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:10.824 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:10.822+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:10.829 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:10.826+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:11.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:11.950+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:11.982 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:11.974+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:12.035 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:12.026+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:12.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:12.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:12.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:12.030+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:12.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:12.030+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:12.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:12.030+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:12.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:12.034+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:14.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:14.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:14.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:14.042+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:14.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:14.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:14.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:14.058+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:14.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:14.058+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:19.905 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.901+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:31:19.906 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.901+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:31:19.928 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:19.937 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:19.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:19.951 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:19.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.945+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:19.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:19.945+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:22.343 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:22.337+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:23.377 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:31:24.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:24.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:24.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:24.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:24.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:24.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.085+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:24.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.085+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:24.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:24.121+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:24.437 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:31:24.438 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:31:24.496 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2892,"stamp":"2023-10-14T01:31:22.998482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29376056908,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2540659,"num_read_kb":9928815,"num_write":2507214,"num_write_kb":10028645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80560,"ondisk_log_size":80560,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24096516,"kb_used_data":20053228,"kb_used_omap":2096,"kb_used_meta":4041167,"kb_avail":259019004,"statfs":{"total":289910292480,"available":265235460096,"internally_reserved":0,"allocated":20534505472,"data_stored":30409538065,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":4138155041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,10,7,15,15,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19378176,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4668,"num_read_kb":18596,"num_write":4716,"num_write_kb":18864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001978"},"pg_stats":[{"pgid":"2.7","version":"141'294193","reported_seq":585137,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:31:19.872009+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:31:19.872009+0000","last_peered":"2023-10-14T01:31:19.872009+0000","last_clean":"2023-10-14T01:31:19.872009+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:31:19.872009+0000","last_undegraded":"2023-10-14T01:31:19.872009+0000","last_fullsized":"2023-10-14T01:31:19.872009+0000","mapping_epoch":13,"log_start":"138'284112","ondisk_log_start":"138'284112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":966,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 285s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3418824887,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":291639,"num_read_kb":1151236,"num_write":292478,"num_write_kb":1169897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"141'309717","reported_seq":610793,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:31:19.870955+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:31:19.870955+0000","last_peered":"2023-10-14T01:31:19.870955+0000","last_clean":"2023-10-14T01:31:19.870955+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:31:19.870955+0000","last_undegraded":"2023-10-14T01:31:19.870955+0000","last_fullsized":"2023-10-14T01:31:19.870955+0000","mapping_epoch":13,"log_start":"138'299710","ondisk_log_start":"138'299710","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":915,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 290s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3447619851,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":302736,"num_read_kb":1173065,"num_write":293956,"num_write_kb":1175423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"141'316402","reported_seq":627832,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:19.871595+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:31:19.871595+0000","last_peered":"2023-10-14T01:31:19.871595+0000","last_clean":"2023-10-14T01:31:19.871595+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:31:19.871595+0000","last_undegraded":"2023-10-14T01:31:19.871595+0000","last_fullsized":"2023-10-14T01:31:19.871595+0000","mapping_epoch":13,"log_start":"138'306314","ondisk_log_start":"138'306314","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3624100395,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":311914,"num_read_kb":1227554,"num_write":309381,"num_write_kb":1237403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"141'334994","reported_seq":657914,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:19.847408+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:31:19.847408+0000","last_peered":"2023-10-14T01:31:19.847408+0000","last_clean":"2023-10-14T01:31:19.847408+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:31:19.847408+0000","last_undegraded":"2023-10-14T01:31:19.847408+0000","last_fullsized":"2023-10-14T01:31:19.847408+0000","mapping_epoch":13,"log_start":"139'324913","ondisk_log_start":"139'324913","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3725865391,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":323281,"num_read_kb":1259415,"num_write":318741,"num_write_kb":1274646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"141'321109","reported_seq":641950,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:18.048403+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:31:18.048403+0000","last_peered":"2023-10-14T01:31:18.048403+0000","last_clean":"2023-10-14T01:31:18.048403+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:31:18.048403+0000","last_undegraded":"2023-10-14T01:31:18.048403+0000","last_fullsized":"2023-10-14T01:31:18.048403+0000","mapping_epoch":13,"log_start":"138'311012","ondisk_log_start":"138'311012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3668099543,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":320485,"num_read_kb":1241191,"num_write":313391,"num_write_kb":1253057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"141'338216","reported_seq":672441,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:22.440890+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:31:22.440890+0000","last_peered":"2023-10-14T01:31:22.440890+0000","last_clean":"2023-10-14T01:31:22.440890+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:31:22.440890+0000","last_undegraded":"2023-10-14T01:31:22.440890+0000","last_fullsized":"2023-10-14T01:31:22.440890+0000","mapping_epoch":13,"log_start":"139'328212","ondisk_log_start":"139'328212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3896222116,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":334497,"num_read_kb":1311864,"num_write":331491,"num_write_kb":1325790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"141'334586","reported_seq":663653,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:22.510258+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:31:22.510258+0000","last_peered":"2023-10-14T01:31:22.510258+0000","last_clean":"2023-10-14T01:31:22.510258+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:31:22.510258+0000","last_undegraded":"2023-10-14T01:31:22.510258+0000","last_fullsized":"2023-10-14T01:31:22.510258+0000","mapping_epoch":13,"log_start":"139'324515","ondisk_log_start":"139'324515","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3842994529,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":329386,"num_read_kb":1292238,"num_write":327858,"num_write_kb":1311242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"141'340621","reported_seq":667158,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:19.856583+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:31:19.856583+0000","last_peered":"2023-10-14T01:31:19.856583+0000","last_clean":"2023-10-14T01:31:19.856583+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:31:19.856583+0000","last_undegraded":"2023-10-14T01:31:19.856583+0000","last_fullsized":"2023-10-14T01:31:19.856583+0000","mapping_epoch":13,"log_start":"139'330617","ondisk_log_start":"139'330617","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3751739828,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":326595,"num_read_kb":1272143,"num_write":319749,"num_write_kb":1278639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":355,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:46.977403+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:29:46.977403+0000","last_peered":"2023-10-14T01:29:46.977403+0000","last_clean":"2023-10-14T01:29:46.977403+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:29:46.977403+0000","last_undegraded":"2023-10-14T01:29:46.977403+0000","last_fullsized":"2023-10-14T01:29:46.977403+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29375466540,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2540533,"num_read_kb":9928706,"num_write":2507045,"num_write_kb":10026097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20531617792,"data_stored":30407702616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80433,"ondisk_log_size":80433,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739464,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4890996,"kb_used_data":3917292,"kb_used_omap":534,"kb_used_meta":973161,"kb_avail":89480844,"statfs":{"total":96636764160,"available":91628384256,"internally_reserved":0,"allocated":4011307008,"data_stored":5939605132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":996517028},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,6,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.58899999999999997},"min":{"1min":0.193,"5min":0.193,"15min":0.182},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.0940000000000001},"min":{"1min":0.217,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":4.423,"5min":10.692,"15min":11.481999999999999},"last":0.28000000000000003}]},{"osd":1,"last update":"Sat Oct 14 01:31:04 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.3919999999999999,"15min":1.7889999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":12.32,"5min":12.32,"15min":44.308},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56200000000000006,"15min":1.546},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":5.266,"5min":5.266,"15min":44.350000000000001},"last":0.38200000000000001}]}]},{"osd":1,"up_from":8,"seq":34359739452,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11364236,"kb_used_data":10026372,"kb_used_omap":761,"kb_used_meta":1337094,"kb_avail":83007604,"statfs":{"total":96636764160,"available":84999786496,"internally_reserved":0,"allocated":10267004928,"data_stored":15204874271,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":1369185018},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,5,4,8,11,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":1.1499999999999999,"15min":0.91900000000000004},"min":{"1min":0.23100000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":1.0449999999999999,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.71799999999999997,"15min":0.97499999999999998},"min":{"1min":0.23699999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":1.482,"5min":16.52,"15min":16.52},"last":0.61399999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:30:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.80800000000000005,"15min":0.98599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.21299999999999999},"max":{"1min":6.5789999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.224,"5min":0.193,"15min":0.193},"max":{"1min":2.206,"5min":6.8369999999999997,"15min":10.308},"last":0.378}]}]},{"osd":0,"up_from":8,"seq":34359739465,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7841284,"kb_used_data":6109564,"kb_used_omap":800,"kb_used_meta":1730911,"kb_avail":86530556,"statfs":{"total":96636764160,"available":88607289344,"internally_reserved":0,"allocated":6256193536,"data_stored":9265058662,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1772452995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":1.4490000000000001,"15min":1.3029999999999999},"min":{"1min":0.19400000000000001,"5min":0.191,"15min":0.183},"max":{"1min":5.3760000000000003,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":2.3730000000000002,"5min":1.5329999999999999,"15min":1.1499999999999999},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":9.9450000000000003,"5min":13.875,"15min":19.376000000000001},"last":0.52100000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:30:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":0.77600000000000002,"15min":0.73899999999999999},"min":{"1min":0.252,"5min":0.185,"15min":0.185},"max":{"1min":5.4580000000000002,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.94499999999999995,"15min":0.86299999999999999},"min":{"1min":0.23599999999999999,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":4.4630000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.27300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6255497216,"data_stored":9264535411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10265915392,"data_stored":15203957804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4010205184,"data_stored":5939209401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:31:24.498 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:31:24.498 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:31:24.499 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:31:25.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.601+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:31:25.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:25.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.605+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:25.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:25.622 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.617+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:25.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.617+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:25.623 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.617+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:25.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:25.661+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:27.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.261+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:31:27.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:27.320 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:27.366 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:27.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.361+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:27.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.361+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:27.367 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.361+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:27.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.377+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:27.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.389+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:31:27.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.389+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:31:27.412 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.409+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:31:27.451 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:27.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:27.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:27.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:27.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:27.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:27.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:28.633 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:28.629+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:31:28.655 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:28.649+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:28.655 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:28.649+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:28.952 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:28.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:28.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:28.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:28.956 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:28.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:28.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:28.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:28.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:28.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:28.957 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:28.953+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:28.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:28.953+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:32.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:32.445+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:32.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:32.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:32.494 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:32.485+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:32.533 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:32.525+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:33.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:33.148 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:33.151 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:33.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:33.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:33.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:33.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.149+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:33.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.185+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:33.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:33.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:33.780 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:33.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:33.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.773+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:33.781 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.773+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:33.782 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:33.773+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:34.921 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:34.917+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:31:34.957 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:34.953+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:37.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.033+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:31:37.037 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.033+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:31:37.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.037+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:31:37.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.089+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:37.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.141+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:37.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.145+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:37.170 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:37.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.165+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:37.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.165+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:37.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.165+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:37.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.177+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:37.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:37.513 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.509+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:37.531 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.525+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:37.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:37.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.557+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:37.563 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.557+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:37.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.557+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:37.610 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:37.605+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:40.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:40.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:40.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:40.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:40.262 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:40.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:40.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:40.257+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:40.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:40.257+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:40.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:40.257+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:40.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:40.281+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:40.518 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:40.513+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:41.726 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:41.721+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:43.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.457+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:43.520 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.517+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:43.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:43.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:43.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.545+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:43.548 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.545+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:43.552 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:43.745 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:43.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:43.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:43.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:43.776 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.769+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:43.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.769+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:43.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.769+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:43.777 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:43.769+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:44.096 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:44.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:44.135 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.129+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:44.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:44.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:44.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.133+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:44.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.133+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:44.144 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.137+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:44.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.261+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:44.306 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:44.343 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:44.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.337+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:44.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.337+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:44.344 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.337+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:44.358 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:44.353+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:44.501 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:31:45.238 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:31:45.239 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:31:45.277 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2904,"stamp":"2023-10-14T01:31:45.004279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29411413580,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2548780,"num_read_kb":9961215,"num_write":2515844,"num_write_kb":10063165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80568,"ondisk_log_size":80568,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24400128,"kb_used_data":20123816,"kb_used_omap":2096,"kb_used_meta":4274191,"kb_avail":258715392,"statfs":{"total":289910292480,"available":264924561408,"internally_reserved":0,"allocated":20606787584,"data_stored":30481820177,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":4376771617},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,6,7,9,7,25,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17035264,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3895,"num_read_kb":15548,"num_write":4168,"num_write_kb":16672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001611"},"pg_stats":[{"pgid":"2.7","version":"141'295216","reported_seq":587057,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:31:40.536037+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:31:40.536037+0000","last_peered":"2023-10-14T01:31:40.536037+0000","last_clean":"2023-10-14T01:31:40.536037+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:31:40.536037+0000","last_undegraded":"2023-10-14T01:31:40.536037+0000","last_fullsized":"2023-10-14T01:31:40.536037+0000","mapping_epoch":13,"log_start":"139'285212","ondisk_log_start":"139'285212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1056,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 306s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3422761143,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":292534,"num_read_kb":1154816,"num_write":293434,"num_write_kb":1173721,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"141'310774","reported_seq":612807,"reported_epoch":141,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:31:40.514948+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:31:40.514948+0000","last_peered":"2023-10-14T01:31:40.514948+0000","last_clean":"2023-10-14T01:31:40.514948+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:31:40.514948+0000","last_undegraded":"2023-10-14T01:31:40.514948+0000","last_fullsized":"2023-10-14T01:31:40.514948+0000","mapping_epoch":13,"log_start":"139'300711","ondisk_log_start":"139'300711","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":955,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 311s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3451695371,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":303680,"num_read_kb":1176833,"num_write":294950,"num_write_kb":1179399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"141'317507","reported_seq":629896,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:40.500530+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:31:40.500530+0000","last_peered":"2023-10-14T01:31:40.500530+0000","last_clean":"2023-10-14T01:31:40.500530+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:31:40.500530+0000","last_undegraded":"2023-10-14T01:31:40.500530+0000","last_fullsized":"2023-10-14T01:31:40.500530+0000","mapping_epoch":13,"log_start":"139'307414","ondisk_log_start":"139'307414","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3628573227,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":312874,"num_read_kb":1231386,"num_write":310475,"num_write_kb":1241779,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"141'336083","reported_seq":659967,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:40.410987+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:31:40.410987+0000","last_peered":"2023-10-14T01:31:40.410987+0000","last_clean":"2023-10-14T01:31:40.410987+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:31:40.410987+0000","last_undegraded":"2023-10-14T01:31:40.410987+0000","last_fullsized":"2023-10-14T01:31:40.410987+0000","mapping_epoch":13,"log_start":"139'326013","ondisk_log_start":"139'326013","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3730162095,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":324233,"num_read_kb":1263195,"num_write":319794,"num_write_kb":1278858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"141'322452","reported_seq":644517,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:43.256754+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:31:43.256754+0000","last_peered":"2023-10-14T01:31:43.256754+0000","last_clean":"2023-10-14T01:31:43.256754+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:31:43.256754+0000","last_undegraded":"2023-10-14T01:31:43.256754+0000","last_fullsized":"2023-10-14T01:31:43.256754+0000","mapping_epoch":13,"log_start":"139'312412","ondisk_log_start":"139'312412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3673461207,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":321683,"num_read_kb":1245979,"num_write":314696,"num_write_kb":1258277,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"141'339327","reported_seq":674654,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:42.719381+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:31:42.719381+0000","last_peered":"2023-10-14T01:31:42.719381+0000","last_clean":"2023-10-14T01:31:42.719381+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:31:42.719381+0000","last_undegraded":"2023-10-14T01:31:42.719381+0000","last_fullsized":"2023-10-14T01:31:42.719381+0000","mapping_epoch":13,"log_start":"139'329312","ondisk_log_start":"139'329312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3900719524,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":335588,"num_read_kb":1316216,"num_write":332594,"num_write_kb":1330202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"141'335698","reported_seq":665796,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:42.680660+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:31:42.680660+0000","last_peered":"2023-10-14T01:31:42.680660+0000","last_clean":"2023-10-14T01:31:42.680660+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:31:42.680660+0000","last_undegraded":"2023-10-14T01:31:42.680660+0000","last_fullsized":"2023-10-14T01:31:42.680660+0000","mapping_epoch":13,"log_start":"139'325615","ondisk_log_start":"139'325615","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3847545185,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":330408,"num_read_kb":1296314,"num_write":328962,"num_write_kb":1315658,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"141'341690","reported_seq":669302,"reported_epoch":141,"state":"active+clean","last_fresh":"2023-10-14T01:31:40.507356+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:31:40.507356+0000","last_peered":"2023-10-14T01:31:40.507356+0000","last_clean":"2023-10-14T01:31:40.507356+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:31:40.507356+0000","last_undegraded":"2023-10-14T01:31:40.507356+0000","last_fullsized":"2023-10-14T01:31:40.507356+0000","mapping_epoch":13,"log_start":"139'331617","ondisk_log_start":"139'331617","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3755905460,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":327654,"num_read_kb":1276367,"num_write":320770,"num_write_kb":1282723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":355,"reported_epoch":140,"state":"active+clean","last_fresh":"2023-10-14T01:29:46.977403+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:29:46.977403+0000","last_peered":"2023-10-14T01:29:46.977403+0000","last_clean":"2023-10-14T01:29:46.977403+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:29:46.977403+0000","last_undegraded":"2023-10-14T01:29:46.977403+0000","last_fullsized":"2023-10-14T01:29:46.977403+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29410823212,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2548654,"num_read_kb":9961106,"num_write":2515675,"num_write_kb":10060617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20605071360,"data_stored":30481156184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80441,"ondisk_log_size":80441,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739468,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":4935144,"kb_used_data":3929504,"kb_used_omap":534,"kb_used_meta":1005097,"kb_avail":89436696,"statfs":{"total":96636764160,"available":91583176704,"internally_reserved":0,"allocated":4023812096,"data_stored":5952110220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1029219492},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.58899999999999997},"min":{"1min":0.193,"5min":0.193,"15min":0.182},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.0940000000000001},"min":{"1min":0.217,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":4.423,"5min":10.692,"15min":11.481999999999999},"last":0.40899999999999997}]},{"osd":1,"last update":"Sat Oct 14 01:31:04 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.3919999999999999,"15min":1.7889999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":12.32,"5min":12.32,"15min":44.308},"last":7.8440000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56200000000000006,"15min":1.546},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":5.266,"5min":5.266,"15min":44.350000000000001},"last":5.7039999999999997}]}]},{"osd":1,"up_from":8,"seq":34359739456,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11557312,"kb_used_data":10060024,"kb_used_omap":761,"kb_used_meta":1496518,"kb_avail":82814528,"statfs":{"total":96636764160,"available":84802076672,"internally_reserved":0,"allocated":10301464576,"data_stored":15239333919,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":1532435194},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,6,6,4,19,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.93200000000000005,"15min":0.81100000000000005},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":1.728,"5min":16.481000000000002,"15min":16.481000000000002},"last":5.7039999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.65900000000000003,"15min":0.86899999999999999},"min":{"1min":0.26000000000000001,"5min":0.22800000000000001,"15min":0.20499999999999999},"max":{"1min":2.032,"5min":16.52,"15min":16.52},"last":5.5069999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:30:40 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.80800000000000005,"15min":0.98599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.21299999999999999},"max":{"1min":6.5789999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":5.8209999999999997},{"interface":"front","average":{"1min":0.438,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.224,"5min":0.193,"15min":0.193},"max":{"1min":2.206,"5min":6.8369999999999997,"15min":10.308},"last":2.5329999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739470,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7907672,"kb_used_data":6134288,"kb_used_omap":800,"kb_used_meta":1772575,"kb_avail":86464168,"statfs":{"total":96636764160,"available":88539308032,"internally_reserved":0,"allocated":6281510912,"data_stored":9290376038,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1815116931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":1.1579999999999999,"15min":1.3029999999999999},"min":{"1min":0.26100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":3.5019999999999998,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.26400000000000001},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.5740000000000001,"15min":1.1639999999999999},"min":{"1min":0.24399999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":5.5300000000000002,"5min":13.875,"15min":19.376000000000001},"last":0.33000000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.22700000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.496,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.39700000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.97799999999999998,"15min":0.83899999999999997},"min":{"1min":0.253,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":5.0190000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":0.36799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6281277440,"data_stored":9290315635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10300477440,"data_stored":15238519852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4023316480,"data_stored":5952320697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:31:45.279 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:31:45.279 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:31:45.280 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:31:46.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:46.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:46.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:46.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:46.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:46.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:46.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:46.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:46.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:46.097+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:46.109 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:46.097+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:46.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:46.117+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:47.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:47.993+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:31:48.000 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:47.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:48.051 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:48.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:48.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:48.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:48.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:48.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:48.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:48.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:48.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:48.073+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:48.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:48.073+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:48.110 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:48.105+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:49.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:49.789+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:50.864 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:50.857+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:53.450 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:53.471 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.461+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:53.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:53.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:53.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.497+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:53.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.497+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:53.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.497+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:53.509 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.497+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:54.004 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:54.005 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:53.993+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:54.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.021+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:54.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:54.045 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.037+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:54.046 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.037+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:54.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.041+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:54.421 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.417+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:54.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:54.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:54.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:54.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:54.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:54.432 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.421+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:54.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.665+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:54.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.729+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:54.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.757+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:54.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:54.799 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.761+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:54.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.761+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:54.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.761+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:54.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:54.789+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:55.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:55.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:55.230 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:55.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:55.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:55.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:55.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:55.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:55.231 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:55.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:55.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:55.221+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:55.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:55.221+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:55.473 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:55.469+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:55.497 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:55.481+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:55.661 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:55.657+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:56.133 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:56.129+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:56.143 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:56.137+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:31:57.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:57.496+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:58.728 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:31:58.724+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:31:58.819 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.812+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:58.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.828+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:58.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:31:58.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:58.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:58.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.856+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:58.863 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.856+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:31:58.876 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:58.872+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:31:59.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:59.952+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:31:59.970 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:59.964+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:31:59.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:59.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:31:59.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:59.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:31:59.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:59.984+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:31:59.992 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:31:59.984+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:00.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:00.272+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:00.294 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:00.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:00.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:00.328+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:00.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:00.336+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:00.345 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:00.340+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:00.351 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:00.344+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:00.356 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:00.352+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:02.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.040+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:02.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:02.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:02.066 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.060+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:02.067 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.060+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:02.075 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.068+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:02.783 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:02.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:02.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.788+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:02.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.788+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:02.796 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:02.788+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:05.281 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:32:06.128 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:32:06.129 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:32:06.185 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2914,"stamp":"2023-10-14T01:32:05.011357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29445496396,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2556737,"num_read_kb":9993003,"num_write":2524170,"num_write_kb":10096469,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80477,"ondisk_log_size":80477,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24854304,"kb_used_data":20192064,"kb_used_omap":2096,"kb_used_meta":4660111,"kb_avail":258261216,"statfs":{"total":289910292480,"available":264459485184,"internally_reserved":0,"allocated":20676673536,"data_stored":30551691723,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":4771953697},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,2,4,11,12,19,14,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18182144,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4328,"num_read_kb":17296,"num_write":4445,"num_write_kb":17780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005677"},"pg_stats":[{"pgid":"2.7","version":"142'296293","reported_seq":589032,"reported_epoch":142,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:32:01.032227+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:32:01.032227+0000","last_peered":"2023-10-14T01:32:01.032227+0000","last_clean":"2023-10-14T01:32:01.032227+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:32:01.032227+0000","last_undegraded":"2023-10-14T01:32:01.032227+0000","last_fullsized":"2023-10-14T01:32:01.032227+0000","mapping_epoch":13,"log_start":"139'286212","ondisk_log_start":"139'286212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1110,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 326s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3427045559,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":293419,"num_read_kb":1158348,"num_write":294473,"num_write_kb":1177877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"142'311828","reported_seq":614806,"reported_epoch":142,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:32:01.045835+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:32:01.045835+0000","last_peered":"2023-10-14T01:32:01.045835+0000","last_clean":"2023-10-14T01:32:01.045835+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:32:01.045835+0000","last_undegraded":"2023-10-14T01:32:01.045835+0000","last_fullsized":"2023-10-14T01:32:01.045835+0000","mapping_epoch":13,"log_start":"139'301811","ondisk_log_start":"139'301811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1008,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 331s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3455648011,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":304595,"num_read_kb":1180485,"num_write":295918,"num_write_kb":1183271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"142'318567","reported_seq":631970,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:01.006462+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:32:01.006462+0000","last_peered":"2023-10-14T01:32:01.006462+0000","last_clean":"2023-10-14T01:32:01.006462+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:32:01.006462+0000","last_undegraded":"2023-10-14T01:32:01.006462+0000","last_fullsized":"2023-10-14T01:32:01.006462+0000","mapping_epoch":13,"log_start":"139'308514","ondisk_log_start":"139'308514","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3632828971,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":313883,"num_read_kb":1235418,"num_write":311519,"num_write_kb":1245955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"142'337214","reported_seq":662147,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:00.978156+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:32:00.978156+0000","last_peered":"2023-10-14T01:32:00.978156+0000","last_clean":"2023-10-14T01:32:00.978156+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:32:00.978156+0000","last_undegraded":"2023-10-14T01:32:00.978156+0000","last_fullsized":"2023-10-14T01:32:00.978156+0000","mapping_epoch":13,"log_start":"139'327213","ondisk_log_start":"139'327213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3734630831,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":325272,"num_read_kb":1267351,"num_write":320877,"num_write_kb":1283190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"142'323456","reported_seq":646518,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:03.525718+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:32:03.525718+0000","last_peered":"2023-10-14T01:32:03.525718+0000","last_clean":"2023-10-14T01:32:03.525718+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:32:03.525718+0000","last_undegraded":"2023-10-14T01:32:03.525718+0000","last_fullsized":"2023-10-14T01:32:03.525718+0000","mapping_epoch":13,"log_start":"139'313412","ondisk_log_start":"139'313412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3677544919,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":322665,"num_read_kb":1249903,"num_write":315695,"num_write_kb":1262273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"142'340405","reported_seq":676771,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:02.978648+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:32:02.978648+0000","last_peered":"2023-10-14T01:32:02.978648+0000","last_clean":"2023-10-14T01:32:02.978648+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:32:02.978648+0000","last_undegraded":"2023-10-14T01:32:02.978648+0000","last_fullsized":"2023-10-14T01:32:02.978648+0000","mapping_epoch":13,"log_start":"139'330312","ondisk_log_start":"139'330312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3905057188,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":336621,"num_read_kb":1320340,"num_write":333658,"num_write_kb":1334458,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"142'336769","reported_seq":667887,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:02.976596+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:32:02.976596+0000","last_peered":"2023-10-14T01:32:02.976596+0000","last_clean":"2023-10-14T01:32:02.976596+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:32:02.976596+0000","last_undegraded":"2023-10-14T01:32:02.976596+0000","last_fullsized":"2023-10-14T01:32:02.976596+0000","mapping_epoch":13,"log_start":"139'326715","ondisk_log_start":"139'326715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3851878753,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":331424,"num_read_kb":1300374,"num_write":330021,"num_write_kb":1319894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"142'342824","reported_seq":671529,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:00.988162+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:32:00.988162+0000","last_peered":"2023-10-14T01:32:00.988162+0000","last_clean":"2023-10-14T01:32:00.988162+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:32:00.988162+0000","last_undegraded":"2023-10-14T01:32:00.988162+0000","last_fullsized":"2023-10-14T01:32:00.988162+0000","mapping_epoch":13,"log_start":"139'332817","ondisk_log_start":"139'332817","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3760271796,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":328732,"num_read_kb":1280675,"num_write":321840,"num_write_kb":1287003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":357,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:31:43.939080+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:31:43.939080+0000","last_peered":"2023-10-14T01:31:43.939080+0000","last_clean":"2023-10-14T01:31:43.939080+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:31:43.939080+0000","last_undegraded":"2023-10-14T01:31:43.939080+0000","last_fullsized":"2023-10-14T01:31:43.939080+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29444906028,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2556611,"num_read_kb":9992894,"num_write":2524001,"num_write_kb":10093921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20673581056,"data_stored":30549665880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80350,"ondisk_log_size":80350,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739472,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5033060,"kb_used_data":3942748,"kb_used_omap":534,"kb_used_meta":1089769,"kb_avail":89338780,"statfs":{"total":96636764160,"available":91482910720,"internally_reserved":0,"allocated":4037373952,"data_stored":5965667274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1115923620},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,2,4,4],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:31:04 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.53100000000000003,"15min":0.58899999999999997},"min":{"1min":0.193,"5min":0.193,"15min":0.182},"max":{"1min":5.3140000000000001,"5min":5.3140000000000001,"15min":5.3140000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":1.149,"5min":1.133,"15min":1.0940000000000001},"min":{"1min":0.217,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":4.423,"5min":10.692,"15min":11.481999999999999},"last":0.20100000000000001}]},{"osd":1,"last update":"Sat Oct 14 01:31:04 2023","interfaces":[{"interface":"back","average":{"1min":1.647,"5min":1.3919999999999999,"15min":1.7889999999999999},"min":{"1min":0.223,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":12.32,"5min":12.32,"15min":44.308},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.56200000000000006,"15min":1.546},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":5.266,"5min":5.266,"15min":44.350000000000001},"last":0.255}]}]},{"osd":1,"up_from":8,"seq":34359739460,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11779812,"kb_used_data":10093720,"kb_used_omap":761,"kb_used_meta":1685318,"kb_avail":82592028,"statfs":{"total":96636764160,"available":84574236672,"internally_reserved":0,"allocated":10335969280,"data_stored":15273833821,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":1725766394},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,2,10,5,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.93200000000000005,"15min":0.81100000000000005},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":1.728,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.65900000000000003,"15min":0.86899999999999999},"min":{"1min":0.26000000000000001,"5min":0.22800000000000001,"15min":0.20499999999999999},"max":{"1min":2.032,"5min":16.52,"15min":16.52},"last":0.51400000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":0.88500000000000001,"15min":0.96799999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":5.8209999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":3.3999999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.193},"max":{"1min":5.3490000000000002,"5min":6.8369999999999997,"15min":10.308},"last":6.2709999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739474,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8041432,"kb_used_data":6155596,"kb_used_omap":800,"kb_used_meta":1885023,"kb_avail":86330408,"statfs":{"total":96636764160,"available":88402337792,"internally_reserved":0,"allocated":6303330304,"data_stored":9312190628,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1930263683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,0,6,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":1.1579999999999999,"15min":1.3029999999999999},"min":{"1min":0.26100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":3.5019999999999998,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.33000000000000002},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.5740000000000001,"15min":1.1639999999999999},"min":{"1min":0.24399999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":5.5300000000000002,"5min":13.875,"15min":19.376000000000001},"last":0.26700000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.22700000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.496,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":0.371},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.97799999999999998,"15min":0.83899999999999997},"min":{"1min":0.253,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":5.0190000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":6.1040000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6302253056,"data_stored":9311291251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10334883840,"data_stored":15272926252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4036444160,"data_stored":5965448377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:32:06.186 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:32:06.187 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:32:06.187 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:32:06.647 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:06.640+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:32:06.665 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:06.656+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:06.665 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:06.656+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:06.871 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:06.864+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:06.892 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:06.888+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:07.615 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:07.608+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:07.632 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:07.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:07.665 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:07.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:07.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:07.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:07.666 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:07.660+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:07.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:07.660+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:07.687 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:07.680+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:09.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.824+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:09.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.848+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:09.862 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.856+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:09.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.872+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:09.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.872+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:09.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.872+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:09.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:09.884+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:12.229 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.224+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:12.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.284+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:12.310 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:12.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.304+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:12.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.304+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:12.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.304+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:12.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.316+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:12.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.348+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:32:12.355 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.348+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:32:12.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.356+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:32:12.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.356+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:12.364 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.360+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:12.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:12.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:12.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.416+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:12.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.416+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:12.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.416+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:12.434 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:12.428+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:13.354 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:13.348+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:13.404 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:13.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:13.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:13.400+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:13.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:13.400+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:13.405 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:13.400+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:13.408 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:13.400+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:14.330 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:14.331 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:14.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.352+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:14.611 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.604+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:14.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:14.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.628+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:14.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.628+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:14.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:14.628+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:15.424 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:15.420+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:15.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:15.564+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:32:15.571 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:15.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:15.582 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:15.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:15.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:15.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:15.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:15.604+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:17.610 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:17.604+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:17.629 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:17.624+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:17.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:17.968+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:17.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:17.984+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:18.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:18.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:18.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:18.016+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:18.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:18.016+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:18.023 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:18.016+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:18.726 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:18.720+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:19.440 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:19.436+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:19.448 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:19.444+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:21.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:21.188+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:21.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:21.204+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:21.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:21.216+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:21.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:21.236+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:21.392 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:21.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:32:21.395 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:21.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:21.395 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:21.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:22.362 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:22.356+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:24.967 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:24.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:24.968 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:24.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:25.020 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:25.008+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:25.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:25.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:25.021 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:25.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:25.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:25.012+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:25.022 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:25.012+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:25.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:25.024+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:26.189 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:32:26.909 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:32:26.910 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:32:26.972 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2924,"stamp":"2023-10-14T01:32:25.018953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29479534156,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2564551,"num_read_kb":10024195,"num_write":2532487,"num_write_kb":10129737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80510,"ondisk_log_size":80510,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25129588,"kb_used_data":20257948,"kb_used_omap":2096,"kb_used_meta":4869519,"kb_avail":257985932,"statfs":{"total":289910292480,"available":264177594368,"internally_reserved":0,"allocated":20744138752,"data_stored":30619156939,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":4986387489},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,0,5,11,7,13,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20328448,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4624,"num_read_kb":18472,"num_write":4961,"num_write_kb":19844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002009"},"pg_stats":[{"pgid":"2.7","version":"142'297262","reported_seq":590915,"reported_epoch":142,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:32:21.517828+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:32:21.517828+0000","last_peered":"2023-10-14T01:32:21.517828+0000","last_clean":"2023-10-14T01:32:21.517828+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:32:21.517828+0000","last_undegraded":"2023-10-14T01:32:21.517828+0000","last_fullsized":"2023-10-14T01:32:21.517828+0000","mapping_epoch":13,"log_start":"139'287212","ondisk_log_start":"139'287212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1188,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 347s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3430764727,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":294324,"num_read_kb":1161960,"num_write":295383,"num_write_kb":1181517,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"142'312901","reported_seq":616804,"reported_epoch":142,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:32:21.528089+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:32:21.528089+0000","last_peered":"2023-10-14T01:32:21.528089+0000","last_clean":"2023-10-14T01:32:21.528089+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:32:21.528089+0000","last_undegraded":"2023-10-14T01:32:21.528089+0000","last_fullsized":"2023-10-14T01:32:21.528089+0000","mapping_epoch":13,"log_start":"139'302811","ondisk_log_start":"139'302811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1086,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 352s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3459633419,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":305499,"num_read_kb":1184085,"num_write":296892,"num_write_kb":1187167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"142'319623","reported_seq":633973,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:21.478234+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:32:21.478234+0000","last_peered":"2023-10-14T01:32:21.478234+0000","last_clean":"2023-10-14T01:32:21.478234+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:32:21.478234+0000","last_undegraded":"2023-10-14T01:32:21.478234+0000","last_fullsized":"2023-10-14T01:32:21.478234+0000","mapping_epoch":13,"log_start":"139'309614","ondisk_log_start":"139'309614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3637109291,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":314822,"num_read_kb":1239170,"num_write":312561,"num_write_kb":1250123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"142'338301","reported_seq":664238,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:21.439292+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:32:21.439292+0000","last_peered":"2023-10-14T01:32:21.439292+0000","last_clean":"2023-10-14T01:32:21.439292+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:32:21.439292+0000","last_undegraded":"2023-10-14T01:32:21.439292+0000","last_fullsized":"2023-10-14T01:32:21.439292+0000","mapping_epoch":13,"log_start":"139'328213","ondisk_log_start":"139'328213","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3738898863,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":326261,"num_read_kb":1271299,"num_write":321922,"num_write_kb":1287370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"142'324495","reported_seq":648545,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:23.683527+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:32:23.683527+0000","last_peered":"2023-10-14T01:32:23.683527+0000","last_clean":"2023-10-14T01:32:23.683527+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:32:23.683527+0000","last_undegraded":"2023-10-14T01:32:23.683527+0000","last_fullsized":"2023-10-14T01:32:23.683527+0000","mapping_epoch":13,"log_start":"139'314412","ondisk_log_start":"139'314412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3681755607,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":323628,"num_read_kb":1253743,"num_write":316724,"num_write_kb":1266389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"142'341543","reported_seq":678987,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:23.296948+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:32:23.296948+0000","last_peered":"2023-10-14T01:32:23.296948+0000","last_clean":"2023-10-14T01:32:23.296948+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:32:23.296948+0000","last_undegraded":"2023-10-14T01:32:23.296948+0000","last_fullsized":"2023-10-14T01:32:23.296948+0000","mapping_epoch":13,"log_start":"139'331512","ondisk_log_start":"139'331512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3909665188,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":337683,"num_read_kb":1324584,"num_write":334778,"num_write_kb":1338938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"142'337846","reported_seq":670029,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:23.308261+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:32:23.308261+0000","last_peered":"2023-10-14T01:32:23.308261+0000","last_clean":"2023-10-14T01:32:23.308261+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:32:23.308261+0000","last_undegraded":"2023-10-14T01:32:23.308261+0000","last_fullsized":"2023-10-14T01:32:23.308261+0000","mapping_epoch":13,"log_start":"139'327815","ondisk_log_start":"139'327815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3856212321,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":332477,"num_read_kb":1304578,"num_write":331080,"num_write_kb":1324130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"142'344018","reported_seq":673724,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:21.438748+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:32:21.438748+0000","last_peered":"2023-10-14T01:32:21.438748+0000","last_clean":"2023-10-14T01:32:21.438748+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:32:21.438748+0000","last_undegraded":"2023-10-14T01:32:21.438748+0000","last_fullsized":"2023-10-14T01:32:21.438748+0000","mapping_epoch":13,"log_start":"139'334017","ondisk_log_start":"139'334017","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3764904372,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":329731,"num_read_kb":1284667,"num_write":322978,"num_write_kb":1291555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":357,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:31:43.939080+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:31:43.939080+0000","last_peered":"2023-10-14T01:31:43.939080+0000","last_clean":"2023-10-14T01:31:43.939080+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:31:43.939080+0000","last_undegraded":"2023-10-14T01:31:43.939080+0000","last_fullsized":"2023-10-14T01:31:43.939080+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29478943788,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2564425,"num_read_kb":10024086,"num_write":2532318,"num_write_kb":10127189,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20741640192,"data_stored":30617725016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80383,"ondisk_log_size":80383,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739476,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5077996,"kb_used_data":3955748,"kb_used_omap":534,"kb_used_meta":1121705,"kb_avail":89293844,"statfs":{"total":96636764160,"available":91436896256,"internally_reserved":0,"allocated":4050685952,"data_stored":5978979274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1148626084},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,2,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":0.76600000000000001,"15min":0.60699999999999998},"min":{"1min":0.20100000000000001,"5min":0.193,"15min":0.182},"max":{"1min":10.244,"5min":10.244,"15min":10.244},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.978,"5min":1.274,"15min":1.1599999999999999},"min":{"1min":0.20100000000000001,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":16.097999999999999,"5min":16.097999999999999,"15min":16.097999999999999},"last":0.625}]},{"osd":1,"last update":"Sat Oct 14 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":3.0190000000000001,"5min":1.6419999999999999,"15min":1.879},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":20.43,"5min":20.43,"15min":44.308},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.8720000000000001,"5min":0.77300000000000002,"15min":1.556},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":10.471,"5min":10.471,"15min":44.350000000000001},"last":0.81899999999999995}]}]},{"osd":1,"up_from":8,"seq":34359739464,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11924268,"kb_used_data":10127140,"kb_used_omap":761,"kb_used_meta":1796358,"kb_avail":82447572,"statfs":{"total":96636764160,"available":84426313728,"internally_reserved":0,"allocated":10370191360,"data_stored":15308055901,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":1839471354},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,6,4,8,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:31:36 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.93200000000000005,"15min":0.81100000000000005},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":1.728,"5min":16.481000000000002,"15min":16.481000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.65900000000000003,"15min":0.86899999999999999},"min":{"1min":0.26000000000000001,"5min":0.22800000000000001,"15min":0.20499999999999999},"max":{"1min":2.032,"5min":16.52,"15min":16.52},"last":0.37}]},{"osd":2,"last update":"Sat Oct 14 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":0.88500000000000001,"15min":0.96799999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":5.8209999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.193},"max":{"1min":5.3490000000000002,"5min":6.8369999999999997,"15min":10.308},"last":0.32800000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739478,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8127324,"kb_used_data":6175060,"kb_used_omap":800,"kb_used_meta":1951455,"kb_avail":86244516,"statfs":{"total":96636764160,"available":88314384384,"internally_reserved":0,"allocated":6323261440,"data_stored":9332121764,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":1998290051},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,0,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":1.1579999999999999,"15min":1.3029999999999999},"min":{"1min":0.26100000000000001,"5min":0.191,"15min":0.191},"max":{"1min":3.5019999999999998,"5min":10.837999999999999,"15min":19.274000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.6240000000000001,"5min":1.5740000000000001,"15min":1.1639999999999999},"min":{"1min":0.24399999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":5.5300000000000002,"5min":13.875,"15min":19.376000000000001},"last":0.20000000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:31:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.71299999999999997,"15min":0.70799999999999996},"min":{"1min":0.22700000000000001,"5min":0.185,"15min":0.185},"max":{"1min":0.496,"5min":7.1319999999999997,"15min":9.3309999999999995},"last":1.02},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.97799999999999998,"15min":0.83899999999999997},"min":{"1min":0.253,"5min":0.23200000000000001,"15min":0.214},"max":{"1min":5.0190000000000001,"5min":7.6289999999999996,"15min":7.6289999999999996},"last":3.601}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6322638848,"data_stored":9331677043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10369040384,"data_stored":15307082796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4049960960,"data_stored":5978965177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:32:26.973 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:32:26.973 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:32:26.974 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:32:32.380 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:32.380+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:32:32.389 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:32.388+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:32.398 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:32.396+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:32.727 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:32.724+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:32.999 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:32.996+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:33.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:33.056+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:33.068 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:33.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:33.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:33.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:33.073 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:33.072+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:33.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:33.072+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:33.074 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:33.072+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:33.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:33.080+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:34.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:34.640+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:34.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:34.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:34.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:34.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:35.648 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:35.643+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:35.729 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:35.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:32:35.730 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:35.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:32:35.730 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:35.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:35.730 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:35.727+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:39.062 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.055+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:32:39.064 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.063+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:32:39.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:39.087 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:39.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:39.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.087+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:39.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.087+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:39.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.087+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:39.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:39.095+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:41.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.111+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:41.129 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:41.149 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.147+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:41.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:41.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.163+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:41.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.163+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:41.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.163+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:41.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.183+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:41.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:41.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:41.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:41.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:41.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:41.203+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:42.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.059+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:32:42.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:42.077 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.071+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:42.111 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:42.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:42.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:42.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.127+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:42.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.127+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:42.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:42.183+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:43.649 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.647+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:43.681 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:43.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:43.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:43.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:43.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.687+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:43.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.687+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:43.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:43.687+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:44.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.079+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:44.108 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:32:44.107+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:32:44.108 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:32:44.107+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:32:44.147 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.139+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:44.455 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:44.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.479+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:44.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:44.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:44.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.499+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:44.502 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.499+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:44.527 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:44.515+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:46.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:46.214 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:46.225 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.223+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:46.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:46.257 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:46.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.255+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:46.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.255+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:46.301 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.295+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:46.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:46.603+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:46.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:32:47.890 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:47.887+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:32:47.904 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:32:47.904 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:32:47.927 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:47.923+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:47.960 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2935,"stamp":"2023-10-14T01:32:47.034474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29513576012,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2572832,"num_read_kb":10057211,"num_write":2540795,"num_write_kb":10162969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80525,"ondisk_log_size":80525,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25532032,"kb_used_data":20322852,"kb_used_omap":2096,"kb_used_meta":5207055,"kb_avail":257583488,"statfs":{"total":289910292480,"available":263765491712,"internally_reserved":0,"allocated":20810600448,"data_stored":30685618635,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2147295,"internal_metadata":5332024353},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,7,6,6,18,13],"upper_bound":256},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":16519168,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4097,"num_read_kb":16328,"num_write":4024,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002056"},"pg_stats":[{"pgid":"2.7","version":"142'298266","reported_seq":592894,"reported_epoch":142,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:32:42.241049+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:32:42.241049+0000","last_peered":"2023-10-14T01:32:42.241049+0000","last_clean":"2023-10-14T01:32:42.241049+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:32:42.241049+0000","last_undegraded":"2023-10-14T01:32:42.241049+0000","last_fullsized":"2023-10-14T01:32:42.241049+0000","mapping_epoch":13,"log_start":"139'288212","ondisk_log_start":"139'288212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1246,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 367s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3434725559,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":295289,"num_read_kb":1165812,"num_write":296346,"num_write_kb":1185369,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"142'313984","reported_seq":618896,"reported_epoch":142,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:32:42.249409+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:32:42.249409+0000","last_peered":"2023-10-14T01:32:42.249409+0000","last_clean":"2023-10-14T01:32:42.249409+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:32:42.249409+0000","last_undegraded":"2023-10-14T01:32:42.249409+0000","last_fullsized":"2023-10-14T01:32:42.249409+0000","mapping_epoch":13,"log_start":"140'303911","ondisk_log_start":"140'303911","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1151,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 372s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3463835915,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":306490,"num_read_kb":1188033,"num_write":297911,"num_write_kb":1191243,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"142'320638","reported_seq":636019,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:42.200150+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:32:42.200150+0000","last_peered":"2023-10-14T01:32:42.200150+0000","last_clean":"2023-10-14T01:32:42.200150+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:32:42.200150+0000","last_undegraded":"2023-10-14T01:32:42.200150+0000","last_fullsized":"2023-10-14T01:32:42.200150+0000","mapping_epoch":13,"log_start":"140'310614","ondisk_log_start":"140'310614","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3641238059,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":315848,"num_read_kb":1243266,"num_write":313569,"num_write_kb":1254155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"142'339387","reported_seq":666410,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:42.188964+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:32:42.188964+0000","last_peered":"2023-10-14T01:32:42.188964+0000","last_clean":"2023-10-14T01:32:42.188964+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:32:42.188964+0000","last_undegraded":"2023-10-14T01:32:42.188964+0000","last_fullsized":"2023-10-14T01:32:42.188964+0000","mapping_epoch":13,"log_start":"140'329313","ondisk_log_start":"140'329313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3743289775,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":327335,"num_read_kb":1275575,"num_write":322990,"num_write_kb":1291642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"142'325539","reported_seq":650565,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:44.064263+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:32:44.064263+0000","last_peered":"2023-10-14T01:32:44.064263+0000","last_clean":"2023-10-14T01:32:44.064263+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:32:44.064263+0000","last_undegraded":"2023-10-14T01:32:44.064263+0000","last_fullsized":"2023-10-14T01:32:44.064263+0000","mapping_epoch":13,"log_start":"140'315512","ondisk_log_start":"140'315512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3685892567,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":324583,"num_read_kb":1257559,"num_write":317738,"num_write_kb":1270445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"142'342665","reported_seq":681227,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:43.586227+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:32:43.586227+0000","last_peered":"2023-10-14T01:32:43.586227+0000","last_clean":"2023-10-14T01:32:43.586227+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:32:43.586227+0000","last_undegraded":"2023-10-14T01:32:43.586227+0000","last_fullsized":"2023-10-14T01:32:43.586227+0000","mapping_epoch":13,"log_start":"140'332612","ondisk_log_start":"140'332612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3914183076,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":338797,"num_read_kb":1329008,"num_write":335890,"num_write_kb":1343386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"142'338888","reported_seq":672108,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:43.558242+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:32:43.558242+0000","last_peered":"2023-10-14T01:32:43.558242+0000","last_clean":"2023-10-14T01:32:43.558242+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:32:43.558242+0000","last_undegraded":"2023-10-14T01:32:43.558242+0000","last_fullsized":"2023-10-14T01:32:43.558242+0000","mapping_epoch":13,"log_start":"140'328815","ondisk_log_start":"140'328815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3860459873,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":333508,"num_read_kb":1308686,"num_write":332112,"num_write_kb":1328258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"142'345137","reported_seq":675992,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:32:42.180629+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:32:42.180629+0000","last_peered":"2023-10-14T01:32:42.180629+0000","last_clean":"2023-10-14T01:32:42.180629+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:32:42.180629+0000","last_undegraded":"2023-10-14T01:32:42.180629+0000","last_fullsized":"2023-10-14T01:32:42.180629+0000","mapping_epoch":13,"log_start":"140'335117","ondisk_log_start":"140'335117","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3769360820,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":330856,"num_read_kb":1289163,"num_write":324070,"num_write_kb":1295923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":357,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:31:43.939080+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:31:43.939080+0000","last_peered":"2023-10-14T01:31:43.939080+0000","last_clean":"2023-10-14T01:31:43.939080+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:31:43.939080+0000","last_undegraded":"2023-10-14T01:31:43.939080+0000","last_fullsized":"2023-10-14T01:31:43.939080+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29512985644,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2572706,"num_read_kb":10057102,"num_write":2540626,"num_write_kb":10160421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20809412608,"data_stored":30685497432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":40732,"internal_metadata":0},"log_size":80398,"ondisk_log_size":80398,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739480,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5168828,"kb_used_data":3968436,"kb_used_omap":534,"kb_used_meta":1199849,"kb_avail":89203012,"statfs":{"total":96636764160,"available":91343884288,"internally_reserved":0,"allocated":4063678464,"data_stored":5991971786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1228645540},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":1.617,"5min":0.76600000000000001,"15min":0.60699999999999998},"min":{"1min":0.20100000000000001,"5min":0.193,"15min":0.182},"max":{"1min":10.244,"5min":10.244,"15min":10.244},"last":0.30399999999999999},{"interface":"front","average":{"1min":1.978,"5min":1.274,"15min":1.1599999999999999},"min":{"1min":0.20100000000000001,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":16.097999999999999,"5min":16.097999999999999,"15min":16.097999999999999},"last":3.488}]},{"osd":1,"last update":"Sat Oct 14 01:32:06 2023","interfaces":[{"interface":"back","average":{"1min":3.0190000000000001,"5min":1.6419999999999999,"15min":1.879},"min":{"1min":0.23999999999999999,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":20.43,"5min":20.43,"15min":44.308},"last":0.36399999999999999},{"interface":"front","average":{"1min":1.8720000000000001,"5min":0.77300000000000002,"15min":1.556},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":10.471,"5min":10.471,"15min":44.350000000000001},"last":0.39900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739468,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12198200,"kb_used_data":10159724,"kb_used_omap":761,"kb_used_meta":2037702,"kb_avail":82173640,"statfs":{"total":96636764160,"available":84145807360,"internally_reserved":0,"allocated":10403557376,"data_stored":15341421917,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":2086607610},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,2,2,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.88500000000000001,"15min":0.84199999999999997},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":8.4469999999999992,"5min":16.329999999999998,"15min":16.481000000000002},"last":0.32700000000000001},{"interface":"front","average":{"1min":1.333,"5min":0.69399999999999995,"15min":0.91100000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":8.407,"5min":8.407,"15min":16.52},"last":0.29699999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:31:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":0.88500000000000001,"15min":0.96799999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":5.8209999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81100000000000005,"15min":0.86899999999999999},"min":{"1min":0.216,"5min":0.216,"15min":0.193},"max":{"1min":5.3490000000000002,"5min":6.8369999999999997,"15min":10.308},"last":0.313}]}]},{"osd":0,"up_from":8,"seq":34359739482,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8165004,"kb_used_data":6194692,"kb_used_omap":800,"kb_used_meta":1969503,"kb_avail":86206836,"statfs":{"total":96636764160,"available":88275800064,"internally_reserved":0,"allocated":6343364608,"data_stored":9352224932,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":820093,"internal_metadata":2016771203},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.86699999999999999,"15min":1.2290000000000001},"min":{"1min":0.21099999999999999,"5min":0.191,"15min":0.191},"max":{"1min":4.3529999999999998,"5min":5.5460000000000003,"15min":10.897},"last":0.374},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.744,"15min":1.155},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":5.6769999999999996,"5min":13.875,"15min":13.875},"last":1.0229999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.623,"15min":0.68999999999999995},"min":{"1min":0.23200000000000001,"5min":0.215,"15min":0.185},"max":{"1min":4.4180000000000001,"5min":5.5949999999999998,"15min":7.1319999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.341,"5min":1.002,"15min":0.88600000000000001},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":6.1040000000000001,"5min":6.5640000000000001,"15min":7.6289999999999996},"last":0.53200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6342963200,"data_stored":9352001395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17032,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10403319808,"data_stored":15341362220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4063129600,"data_stored":5992133817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:32:47.961 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:32:47.962 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:32:47.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.7 2023-10-14T01:32:48.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:48.663+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:48.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:48.671+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:48.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:48.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:48.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:48.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:48.791 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.7 on osd.1 to deep-scrub 2023-10-14T01:32:48.826 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 2.6 2023-10-14T01:32:49.239 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.235+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:32:49.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.239+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:49.254 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:49.255 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.251+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:49.270 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:49.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:49.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.267+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:49.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.279+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:49.296 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.295+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:49.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.331+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:49.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:49.339 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.335+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:49.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.359+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:49.363 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:49.359+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:49.631 INFO:teuthology.orchestra.run.smithi183.stderr:instructing pg 2.6 on osd.1 to deep-scrub 2023-10-14T01:32:49.684 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:32:50.558 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:50.601 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.595+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:50.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:50.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:50.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.627+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:50.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.627+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:50.631 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.627+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:50.646 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:50.643+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:51.081 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:51.079+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:51.084 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:51.083+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:52.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:52.031+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:52.097 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:52.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:52.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:52.091+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:52.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:52.091+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:52.099 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:52.091+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:52.333 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:52.331+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:32:52.361 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:52.359+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:52.368 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:52.367+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:52.988 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:52.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:52.989 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:52.987+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:53.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:53.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:53.054 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:53.043+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:53.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:53.047+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:53.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:53.047+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:53.055 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:53.047+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:53.056 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:53.051+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:55.625 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:55.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:55.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:55.651+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:55.677 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:55.675+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:55.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:55.683+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:55.693 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:55.683+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:55.698 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:55.695+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:55.723 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:55.703+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:57.595 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:57.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:32:57.605 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:57.603+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:32:57.623 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:32:57.619+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:32:58.172 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.167+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:58.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:58.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:58.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.215+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:58.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.215+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:58.222 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.215+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:58.228 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.227+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:32:58.872 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.871+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:32:58.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:32:58.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:32:58.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:32:58.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.903+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:32:58.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.903+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:32:58.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.903+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:32:58.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:32:58.903+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:01.149 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:01.147+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:33:01.150 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:01.147+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:01.352 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:01.347+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:03.708 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.699+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:03.714 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.707+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:03.725 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.723+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:03.743 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:03.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.739+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:03.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.739+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:03.744 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.739+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:03.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:03.747+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:04.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:04.819+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:04.843 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:04.835+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:04.880 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:04.879+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:04.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:04.888 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:04.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:04.883+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:04.889 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:04.887+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:05.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:05.187+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:05.258 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:05.251+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:33:05.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:05.251+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:33:05.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:05.255+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:05.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:05.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:05.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:05.263+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:06.972 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:06.967+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:06.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:06.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:06.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:06.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:06.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:06.971+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:06.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:06.971+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:06.980 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:06.971+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:07.025 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:07.019+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:07.205 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:07.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:07.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:07.211+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:07.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:07.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:07.271 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:07.267+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:07.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:07.267+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:07.272 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:07.267+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:07.953 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:07.951+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:08.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.143+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:33:08.153 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.143+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:33:08.160 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.159+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:08.171 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:08.167+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:08.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:08.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.183+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:08.186 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.183+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:08.696 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:08.762 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.755+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:08.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:08.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.775+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:08.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.775+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:08.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.775+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:08.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:08.803+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:09.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:33:10.429 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:33:10.433 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:33:10.483 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2947,"stamp":"2023-10-14T01:33:09.044612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29555523148,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2582880,"num_read_kb":10097255,"num_write":2550994,"num_write_kb":10203765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80627,"ondisk_log_size":80627,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25093892,"kb_used_data":20401768,"kb_used_omap":2086,"kb_used_meta":4690009,"kb_avail":258021628,"statfs":{"total":289910292480,"available":264214147072,"internally_reserved":0,"allocated":20891410432,"data_stored":30766414084,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2136897,"internal_metadata":4802569407},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,3,7,11,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24907776,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5896,"num_read_kb":23496,"num_write":6060,"num_write_kb":24240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004692"},"pg_stats":[{"pgid":"2.7","version":"143'299502","reported_seq":595298,"reported_epoch":143,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:33:08.071874+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:33:08.071874+0000","last_peered":"2023-10-14T01:33:08.071874+0000","last_clean":"2023-10-14T01:33:08.071874+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:33:08.071874+0000","last_undegraded":"2023-10-14T01:33:08.071874+0000","last_fullsized":"2023-10-14T01:33:08.071874+0000","mapping_epoch":13,"log_start":"140'289412","ondisk_log_start":"140'289412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1352,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 393s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3439472823,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":296439,"num_read_kb":1170400,"num_write":297505,"num_write_kb":1190005,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"143'315393","reported_seq":621572,"reported_epoch":143,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:33:08.103226+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:33:08.103226+0000","last_peered":"2023-10-14T01:33:08.103226+0000","last_clean":"2023-10-14T01:33:08.103226+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:33:08.103226+0000","last_undegraded":"2023-10-14T01:33:08.103226+0000","last_fullsized":"2023-10-14T01:33:08.103226+0000","mapping_epoch":13,"log_start":"140'305311","ondisk_log_start":"140'305311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1211,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 398s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3469168907,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":307728,"num_read_kb":1192965,"num_write":299215,"num_write_kb":1196459,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"143'321909","reported_seq":638528,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:08.127347+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:33:08.127347+0000","last_peered":"2023-10-14T01:33:08.127347+0000","last_clean":"2023-10-14T01:33:08.127347+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:33:08.127347+0000","last_undegraded":"2023-10-14T01:33:08.127347+0000","last_fullsized":"2023-10-14T01:33:08.127347+0000","mapping_epoch":13,"log_start":"140'311814","ondisk_log_start":"140'311814","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3646345771,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":317081,"num_read_kb":1248182,"num_write":314819,"num_write_kb":1259155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"143'340875","reported_seq":669227,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:08.100992+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:33:08.100992+0000","last_peered":"2023-10-14T01:33:08.100992+0000","last_clean":"2023-10-14T01:33:08.100992+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:33:08.100992+0000","last_undegraded":"2023-10-14T01:33:08.100992+0000","last_fullsized":"2023-10-14T01:33:08.100992+0000","mapping_epoch":13,"log_start":"140'330813","ondisk_log_start":"140'330813","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3749216687,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":328648,"num_read_kb":1280803,"num_write":324415,"num_write_kb":1297342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"143'326644","reported_seq":652727,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:04.186067+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:33:04.186067+0000","last_peered":"2023-10-14T01:33:04.186067+0000","last_clean":"2023-10-14T01:33:04.186067+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:33:04.186067+0000","last_undegraded":"2023-10-14T01:33:04.186067+0000","last_fullsized":"2023-10-14T01:33:04.186067+0000","mapping_epoch":13,"log_start":"140'316612","ondisk_log_start":"140'316612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3690308055,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":325629,"num_read_kb":1261723,"num_write":318818,"num_write_kb":1274765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"143'344051","reported_seq":683988,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:08.698332+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:33:08.698332+0000","last_peered":"2023-10-14T01:33:08.698332+0000","last_clean":"2023-10-14T01:33:08.698332+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:33:08.698332+0000","last_undegraded":"2023-10-14T01:33:08.698332+0000","last_fullsized":"2023-10-14T01:33:08.698332+0000","mapping_epoch":13,"log_start":"140'334012","ondisk_log_start":"140'334012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3919839652,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":340158,"num_read_kb":1334432,"num_write":337260,"num_write_kb":1348866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"143'340117","reported_seq":674674,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:08.705767+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:33:08.705767+0000","last_peered":"2023-10-14T01:33:08.705767+0000","last_clean":"2023-10-14T01:33:08.705767+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:33:08.705767+0000","last_undegraded":"2023-10-14T01:33:08.705767+0000","last_fullsized":"2023-10-14T01:33:08.705767+0000","mapping_epoch":13,"log_start":"140'330115","ondisk_log_start":"140'330115","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3865481569,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":334831,"num_read_kb":1313966,"num_write":333325,"num_write_kb":1333110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"143'346615","reported_seq":678879,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:08.084647+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:33:08.084647+0000","last_peered":"2023-10-14T01:33:08.084647+0000","last_clean":"2023-10-14T01:33:08.084647+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:33:08.084647+0000","last_undegraded":"2023-10-14T01:33:08.084647+0000","last_fullsized":"2023-10-14T01:33:08.084647+0000","mapping_epoch":13,"log_start":"140'336517","ondisk_log_start":"140'336517","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3775099316,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":332240,"num_read_kb":1294675,"num_write":325468,"num_write_kb":1301515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":358,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:31:50.742161+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:31:50.742161+0000","last_peered":"2023-10-14T01:31:50.742161+0000","last_clean":"2023-10-14T01:31:50.742161+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:31:50.742161+0000","last_undegraded":"2023-10-14T01:31:50.742161+0000","last_fullsized":"2023-10-14T01:31:50.742161+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29554932780,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2582754,"num_read_kb":10097146,"num_write":2550825,"num_write_kb":10201217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20889432064,"data_stored":30765516888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80500,"ondisk_log_size":80500,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739485,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5263224,"kb_used_data":3984816,"kb_used_omap":534,"kb_used_meta":1277865,"kb_avail":89108616,"statfs":{"total":96636764160,"available":91247222784,"internally_reserved":0,"allocated":4080451584,"data_stored":6008740061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1308533924},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.63},"min":{"1min":0.26700000000000002,"5min":0.193,"15min":0.182},"max":{"1min":4.9530000000000003,"5min":10.244,"15min":10.244},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.995,"5min":1.216,"15min":1.1279999999999999},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":3.7570000000000001,"5min":16.097999999999999,"15min":16.097999999999999},"last":3.1269999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.8089999999999999,"15min":1.877},"min":{"1min":0.27600000000000002,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":11.196,"5min":20.43,"15min":44.308},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.77500000000000002,"15min":1.448},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.97099999999999997,"5min":10.471,"15min":44.350000000000001},"last":0.214}]}]},{"osd":1,"up_from":8,"seq":34359739473,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12271192,"kb_used_data":10201356,"kb_used_omap":761,"kb_used_meta":2069062,"kb_avail":82100648,"statfs":{"total":96636764160,"available":84071063552,"internally_reserved":0,"allocated":10446188544,"data_stored":15384048240,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":779526,"internal_metadata":2118720250},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,5,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.88500000000000001,"15min":0.84199999999999997},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":8.4469999999999992,"5min":16.329999999999998,"15min":16.481000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":1.333,"5min":0.69399999999999995,"15min":0.91100000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":8.407,"5min":8.407,"15min":16.52},"last":0.38200000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:32:48 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.93600000000000005,"15min":0.98099999999999998},"min":{"1min":0.25800000000000001,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":4.4539999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.1739999999999999,"5min":0.92800000000000005,"15min":0.86699999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":6.2709999999999999,"5min":6.8369999999999997,"15min":10.308},"last":0.25600000000000001}]}]},{"osd":0,"up_from":8,"seq":34359739486,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7559476,"kb_used_data":6215596,"kb_used_omap":790,"kb_used_meta":1343081,"kb_avail":86812364,"statfs":{"total":96636764160,"available":88895860736,"internally_reserved":0,"allocated":6364770304,"data_stored":9373625783,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":1375315233},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.86699999999999999,"15min":1.2290000000000001},"min":{"1min":0.21099999999999999,"5min":0.191,"15min":0.191},"max":{"1min":4.3529999999999998,"5min":5.5460000000000003,"15min":10.897},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.744,"15min":1.155},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":5.6769999999999996,"5min":13.875,"15min":13.875},"last":4.3070000000000004}]},{"osd":2,"last update":"Sat Oct 14 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.623,"15min":0.68999999999999995},"min":{"1min":0.23200000000000001,"5min":0.215,"15min":0.185},"max":{"1min":4.4180000000000001,"5min":5.5949999999999998,"15min":7.1319999999999997},"last":0.26600000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.002,"15min":0.88600000000000001},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":6.1040000000000001,"5min":6.5640000000000001,"15min":7.6289999999999996},"last":0.308}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6363627520,"data_stored":9372665715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10446589952,"data_stored":15384632364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4079214592,"data_stored":6008218809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:33:10.484 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:33:10.485 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:33:10.485 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:33:11.323 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:11.319+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:12.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.103+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:33:12.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.103+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:12.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:12.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:12.174 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:12.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.171+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:12.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.171+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:12.175 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.171+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:12.176 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:12.171+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:13.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:13.371+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:13.392 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:13.387+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:13.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:13.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:13.460 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:13.459+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:13.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:13.459+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:13.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:13.459+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:13.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:13.459+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:14.091 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:14.083+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:33:14.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:14.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:14.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:14.838+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:15.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.570+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:15.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.582+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:15.590 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:15.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:15.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.610+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:15.612 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.610+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:15.613 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.610+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:15.648 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:15.642+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:16.795 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:16.790+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:33:16.825 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:16.822+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:33:16.841 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:16.838+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:17.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:17.338+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:17.397 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:17.394+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:17.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:17.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:17.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:17.458+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:17.466 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:17.458+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:17.467 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:17.458+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:17.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:17.494+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:18.032 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.022+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:18.040 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.034+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:18.048 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.046+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:18.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:18.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.082+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:18.088 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.082+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:18.089 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.082+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:18.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:18.102+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:19.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:19.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:19.203 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:19.198+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:19.215 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:19.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:19.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:19.210+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:19.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:19.210+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:19.220 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:19.210+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:20.890 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:20.886+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:24.423 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.418+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:24.431 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.426+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:24.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.442+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:24.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:24.510 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.506+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:24.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.506+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:24.511 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.506+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:24.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:24.526+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:26.685 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:26.654+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:26.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:26.690+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:26.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:26.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:26.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:26.710+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:26.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:26.710+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:26.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:26.710+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:26.749 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:26.746+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:27.456 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:27.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:27.479 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:27.474+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:27.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:27.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:27.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:27.482+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:27.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:27.482+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:27.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:27.482+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:27.488 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:27.482+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:28.178 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:28.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:28.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:28.174+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:29.220 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:29.218+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:33:29.226 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:29.222+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:29.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.774+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:29.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:29.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.794+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:29.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:29.804 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.798+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:29.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.798+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:29.805 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.798+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:29.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.814+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:29.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.830+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:29.856 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.850+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:29.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:29.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:29.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.862+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:29.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.862+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:29.865 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:29.862+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:30.486 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:33:31.250 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:33:31.251 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:33:31.293 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2958,"stamp":"2023-10-14T01:33:31.050748+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29591227980,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2591318,"num_read_kb":10130915,"num_write":2559725,"num_write_kb":10238689,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80455,"ondisk_log_size":80455,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25280592,"kb_used_data":20475764,"kb_used_omap":2142,"kb_used_meta":4802657,"kb_avail":257834928,"statfs":{"total":289910292480,"available":264022966272,"internally_reserved":0,"allocated":20967182336,"data_stored":30842185988,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2193702,"internal_metadata":4917921498},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,0,8,11,15,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18366464,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4305,"num_read_kb":17188,"num_write":4478,"num_write_kb":17912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002037"},"pg_stats":[{"pgid":"2.7","version":"143'300520","reported_seq":597228,"reported_epoch":143,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:33:28.684867+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:33:28.684867+0000","last_peered":"2023-10-14T01:33:28.684867+0000","last_clean":"2023-10-14T01:33:28.684867+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:33:28.684867+0000","last_undegraded":"2023-10-14T01:33:28.684867+0000","last_fullsized":"2023-10-14T01:33:28.684867+0000","mapping_epoch":13,"log_start":"140'290513","ondisk_log_start":"140'290513","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1425,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 414s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3443396791,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":297340,"num_read_kb":1174000,"num_write":298467,"num_write_kb":1193853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"143'316438","reported_seq":623653,"reported_epoch":143,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:33:28.677844+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:33:28.677844+0000","last_peered":"2023-10-14T01:33:28.677844+0000","last_clean":"2023-10-14T01:33:28.677844+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:33:28.677844+0000","last_undegraded":"2023-10-14T01:33:28.677844+0000","last_fullsized":"2023-10-14T01:33:28.677844+0000","mapping_epoch":13,"log_start":"140'306411","ondisk_log_start":"140'306411","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1274,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 419s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3473068299,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":308740,"num_read_kb":1196997,"num_write":300170,"num_write_kb":1200279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"143'322984","reported_seq":640645,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:28.677524+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:33:28.677524+0000","last_peered":"2023-10-14T01:33:28.677524+0000","last_clean":"2023-10-14T01:33:28.677524+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:33:28.677524+0000","last_undegraded":"2023-10-14T01:33:28.677524+0000","last_fullsized":"2023-10-14T01:33:28.677524+0000","mapping_epoch":13,"log_start":"140'312914","ondisk_log_start":"140'312914","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3650675243,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":318116,"num_read_kb":1252306,"num_write":315880,"num_write_kb":1263399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"143'342037","reported_seq":671521,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:28.661533+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:33:28.661533+0000","last_peered":"2023-10-14T01:33:28.661533+0000","last_clean":"2023-10-14T01:33:28.661533+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:33:28.661533+0000","last_undegraded":"2023-10-14T01:33:28.661533+0000","last_fullsized":"2023-10-14T01:33:28.661533+0000","mapping_epoch":13,"log_start":"140'332013","ondisk_log_start":"140'332013","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3753804207,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":329769,"num_read_kb":1285271,"num_write":325535,"num_write_kb":1301822,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"143'327991","reported_seq":655389,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:29.429846+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:33:29.429846+0000","last_peered":"2023-10-14T01:33:29.429846+0000","last_clean":"2023-10-14T01:33:29.429846+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:33:29.429846+0000","last_undegraded":"2023-10-14T01:33:29.429846+0000","last_fullsized":"2023-10-14T01:33:29.429846+0000","mapping_epoch":13,"log_start":"140'317912","ondisk_log_start":"140'317912","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3695714775,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":326914,"num_read_kb":1266851,"num_write":320145,"num_write_kb":1280073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"143'345192","reported_seq":686156,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:28.760900+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:33:28.760900+0000","last_peered":"2023-10-14T01:33:28.760900+0000","last_clean":"2023-10-14T01:33:28.760900+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:33:28.760900+0000","last_undegraded":"2023-10-14T01:33:28.760900+0000","last_fullsized":"2023-10-14T01:33:28.760900+0000","mapping_epoch":13,"log_start":"140'335112","ondisk_log_start":"140'335112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3924484516,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":341176,"num_read_kb":1338496,"num_write":338391,"num_write_kb":1353390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"143'341245","reported_seq":676807,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:28.765143+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:33:28.765143+0000","last_peered":"2023-10-14T01:33:28.765143+0000","last_clean":"2023-10-14T01:33:28.765143+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:33:28.765143+0000","last_undegraded":"2023-10-14T01:33:28.765143+0000","last_fullsized":"2023-10-14T01:33:28.765143+0000","mapping_epoch":13,"log_start":"140'331215","ondisk_log_start":"140'331215","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3870036321,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":335833,"num_read_kb":1317966,"num_write":334443,"num_write_kb":1337582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"143'347728","reported_seq":681073,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:28.612566+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:33:28.612566+0000","last_peered":"2023-10-14T01:33:28.612566+0000","last_clean":"2023-10-14T01:33:28.612566+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:33:28.612566+0000","last_undegraded":"2023-10-14T01:33:28.612566+0000","last_fullsized":"2023-10-14T01:33:28.612566+0000","mapping_epoch":13,"log_start":"140'337717","ondisk_log_start":"140'337717","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3779457460,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":333304,"num_read_kb":1298919,"num_write":326525,"num_write_kb":1305743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":358,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:31:50.742161+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:31:50.742161+0000","last_peered":"2023-10-14T01:31:50.742161+0000","last_clean":"2023-10-14T01:31:50.742161+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:31:50.742161+0000","last_undegraded":"2023-10-14T01:31:50.742161+0000","last_fullsized":"2023-10-14T01:31:50.742161+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29590637612,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2591192,"num_read_kb":10130806,"num_write":2559556,"num_write_kb":10236141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20964020224,"data_stored":30840105048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80328,"ondisk_log_size":80328,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5308288,"kb_used_data":3997944,"kb_used_omap":534,"kb_used_meta":1309801,"kb_avail":89063552,"statfs":{"total":96636764160,"available":91201077248,"internally_reserved":0,"allocated":4093894656,"data_stored":6022183133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1341236388},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.63},"min":{"1min":0.26700000000000002,"5min":0.193,"15min":0.182},"max":{"1min":4.9530000000000003,"5min":10.244,"15min":10.244},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.995,"5min":1.216,"15min":1.1279999999999999},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":3.7570000000000001,"5min":16.097999999999999,"15min":16.097999999999999},"last":0.27100000000000002}]},{"osd":1,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.8089999999999999,"15min":1.877},"min":{"1min":0.27600000000000002,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":11.196,"5min":20.43,"15min":44.308},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.77500000000000002,"15min":1.448},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.97099999999999997,"5min":10.471,"15min":44.350000000000001},"last":0.60499999999999998}]}]},{"osd":1,"up_from":8,"seq":34359739477,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12270924,"kb_used_data":10236928,"kb_used_omap":816,"kb_used_meta":2033167,"kb_avail":82100916,"statfs":{"total":96636764160,"available":84071337984,"internally_reserved":0,"allocated":10482614272,"data_stored":15420473968,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":836331,"internal_metadata":2081963285},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,0,4,5,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:32:38 2023","interfaces":[{"interface":"back","average":{"1min":1.0329999999999999,"5min":0.88500000000000001,"15min":0.84199999999999997},"min":{"1min":0.247,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":8.4469999999999992,"5min":16.329999999999998,"15min":16.481000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.333,"5min":0.69399999999999995,"15min":0.91100000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":8.407,"5min":8.407,"15min":16.52},"last":0.69999999999999996}]},{"osd":2,"last update":"Sat Oct 14 01:32:48 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.93600000000000005,"15min":0.98099999999999998},"min":{"1min":0.25800000000000001,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":4.4539999999999997,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":1.1739999999999999,"5min":0.92800000000000005,"15min":0.86699999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":6.2709999999999999,"5min":6.8369999999999997,"15min":10.308},"last":0.33100000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739491,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7701380,"kb_used_data":6240892,"kb_used_omap":790,"kb_used_meta":1459689,"kb_avail":86670460,"statfs":{"total":96636764160,"available":88750551040,"internally_reserved":0,"allocated":6390673408,"data_stored":9399528887,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":1494721825},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.86699999999999999,"15min":1.2290000000000001},"min":{"1min":0.21099999999999999,"5min":0.191,"15min":0.191},"max":{"1min":4.3529999999999998,"5min":5.5460000000000003,"15min":10.897},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.744,"15min":1.155},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":5.6769999999999996,"5min":13.875,"15min":13.875},"last":14.532999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:32:36 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.623,"15min":0.68999999999999995},"min":{"1min":0.23200000000000001,"5min":0.215,"15min":0.185},"max":{"1min":4.4180000000000001,"5min":5.5949999999999998,"15min":7.1319999999999997},"last":0.435},{"interface":"front","average":{"1min":1.341,"5min":1.002,"15min":0.88600000000000001},"min":{"1min":0.251,"5min":0.23599999999999999,"15min":0.214},"max":{"1min":6.1040000000000001,"5min":6.5640000000000001,"15min":7.6289999999999996},"last":0.29399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6389780480,"data_stored":9398818675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10481557504,"data_stored":15419599916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4092682240,"data_stored":6021686457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:33:31.295 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:33:31.295 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:33:31.296 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:33:31.619 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.614+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:31.627 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.622+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:31.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:31.651 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.642+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:31.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.646+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:31.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.646+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:31.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.650+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:31.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:31.650+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:33.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:33.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:33.125 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:33.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:33.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:33.122+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:33.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:33.122+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:33.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:33.122+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:33.161 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:33.158+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:34.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:34.094+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:34.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:34.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:34.156 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:34.150+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:34.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:34.150+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:34.157 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:34.150+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:34.526 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:34.522+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:34.569 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:34.566+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:33:34.583 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:34.574+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:35.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.538+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:35.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.558+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:35.594 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.586+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:35.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:35.602 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.598+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:35.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.598+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:35.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.598+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:35.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:35.634+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:37.161 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:37.158+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:33:37.177 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:37.174+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:37.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.318+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:37.350 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.346+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:37.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:37.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:37.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.378+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:37.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.378+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:37.385 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.378+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:37.394 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:37.386+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:37.907 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:37.902+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:33:37.976 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:37.974+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:33:37.981 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:37.978+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:38.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:38.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:38.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.018+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:38.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.018+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:38.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.018+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:38.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.018+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:38.171 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.166+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:38.191 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.186+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:38.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:38.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.226+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:38.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.226+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:38.240 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.238+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:38.263 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:38.258+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:42.834 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:42.830+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:33:42.869 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:42.866+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:43.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:43.954+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:43.984 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:43.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:43.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:43.982+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:43.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:43.982+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:43.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:43.982+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:44.031 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:44.026+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:44.096 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:33:44.094+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69454680 2023-10-14T01:33:44.097 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:33:44.094+0000 7f1ddcd09640 -1 reset not still connected to 0x561a698a7930 2023-10-14T01:33:44.097 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:33:44.094+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aa5a00 2023-10-14T01:33:44.097 INFO:tasks.ceph.osd.0.smithi183.stderr:2023-10-14T01:33:44.094+0000 7f1ddcd09640 -1 reset not still connected to 0x561a69aff790 2023-10-14T01:33:45.204 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:45.202+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:33:45.205 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:45.202+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:46.629 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.626+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:33:46.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.630+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:46.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.638+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:46.664 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.662+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:46.690 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.686+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:46.756 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:46.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:46.757 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.754+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:46.766 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.762+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:46.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.806+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:46.907 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:46.898+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:48.078 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:33:48.074+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:33:48.198 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:48.194+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:48.223 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:48.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:48.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:48.218+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:48.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:48.218+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:48.224 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:48.218+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:49.118 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.110+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:49.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.130+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:49.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.274+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:49.289 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:49.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:49.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.278+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:49.290 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.278+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:49.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.290+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:49.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.422+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:33:49.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.422+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:33:49.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.438+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:33:49.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.438+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:49.458 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.454+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:49.475 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.470+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:49.484 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:49.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.478+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:49.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.478+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:49.485 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.478+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:49.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.486+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:49.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:49.486+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:51.296 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:33:51.897 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:51.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:51.898 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:51.894+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:51.912 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:51.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:51.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:51.910+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:51.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:51.910+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:51.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:51.910+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:52.026 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:33:52.027 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:33:52.075 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2968,"stamp":"2023-10-14T01:33:51.064795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29625544268,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2599572,"num_read_kb":10163855,"num_write":2568130,"num_write_kb":10272309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80604,"ondisk_log_size":80604,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25490140,"kb_used_data":20542008,"kb_used_omap":2142,"kb_used_meta":4945953,"kb_avail":257625380,"statfs":{"total":289910292480,"available":263808389120,"internally_reserved":0,"allocated":21035016192,"data_stored":30910019844,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2193702,"internal_metadata":5064656602},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,4,12,10,14],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23199744,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5563,"num_read_kb":22196,"num_write":5672,"num_write_kb":22688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009021"},"pg_stats":[{"pgid":"2.7","version":"143'301455","reported_seq":599075,"reported_epoch":143,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:33:49.371096+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:33:49.371096+0000","last_peered":"2023-10-14T01:33:49.371096+0000","last_clean":"2023-10-14T01:33:49.371096+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:33:49.371096+0000","last_undegraded":"2023-10-14T01:33:49.371096+0000","last_fullsized":"2023-10-14T01:33:49.371096+0000","mapping_epoch":13,"log_start":"140'291413","ondisk_log_start":"140'291413","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1490,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 434s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3447009463,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":298241,"num_read_kb":1177592,"num_write":299354,"num_write_kb":1197401,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"143'317531","reported_seq":625695,"reported_epoch":143,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:33:49.392064+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:33:49.392064+0000","last_peered":"2023-10-14T01:33:49.392064+0000","last_clean":"2023-10-14T01:33:49.392064+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:33:49.392064+0000","last_undegraded":"2023-10-14T01:33:49.392064+0000","last_fullsized":"2023-10-14T01:33:49.392064+0000","mapping_epoch":13,"log_start":"141'307511","ondisk_log_start":"141'307511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1350,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 439s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3477123339,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":309672,"num_read_kb":1200725,"num_write":301160,"num_write_kb":1204239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"143'324114","reported_seq":642831,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:49.395361+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:33:49.395361+0000","last_peered":"2023-10-14T01:33:49.395361+0000","last_clean":"2023-10-14T01:33:49.395361+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:33:49.395361+0000","last_undegraded":"2023-10-14T01:33:49.395361+0000","last_fullsized":"2023-10-14T01:33:49.395361+0000","mapping_epoch":13,"log_start":"140'314014","ondisk_log_start":"140'314014","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3655238187,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":319170,"num_read_kb":1256510,"num_write":316993,"num_write_kb":1267851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"143'343101","reported_seq":673668,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:49.365483+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:33:49.365483+0000","last_peered":"2023-10-14T01:33:49.365483+0000","last_clean":"2023-10-14T01:33:49.365483+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:33:49.365483+0000","last_undegraded":"2023-10-14T01:33:49.365483+0000","last_fullsized":"2023-10-14T01:33:49.365483+0000","mapping_epoch":13,"log_start":"141'333013","ondisk_log_start":"141'333013","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3757937071,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":330839,"num_read_kb":1289547,"num_write":326548,"num_write_kb":1305874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"143'329061","reported_seq":657486,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:49.532854+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:33:49.532854+0000","last_peered":"2023-10-14T01:33:49.532854+0000","last_clean":"2023-10-14T01:33:49.532854+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:33:49.532854+0000","last_undegraded":"2023-10-14T01:33:49.532854+0000","last_fullsized":"2023-10-14T01:33:49.532854+0000","mapping_epoch":13,"log_start":"141'319012","ondisk_log_start":"141'319012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3699999191,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":327922,"num_read_kb":1270871,"num_write":321195,"num_write_kb":1284273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"143'346342","reported_seq":688441,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:48.912432+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:33:48.912432+0000","last_peered":"2023-10-14T01:33:48.912432+0000","last_clean":"2023-10-14T01:33:48.912432+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:33:48.912432+0000","last_undegraded":"2023-10-14T01:33:48.912432+0000","last_fullsized":"2023-10-14T01:33:48.912432+0000","mapping_epoch":13,"log_start":"141'336312","ondisk_log_start":"141'336312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3929108900,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":342303,"num_read_kb":1342992,"num_write":339523,"num_write_kb":1357918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"143'342479","reported_seq":679203,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:48.913488+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:33:48.913488+0000","last_peered":"2023-10-14T01:33:48.913488+0000","last_clean":"2023-10-14T01:33:48.913488+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:33:48.913488+0000","last_undegraded":"2023-10-14T01:33:48.913488+0000","last_fullsized":"2023-10-14T01:33:48.913488+0000","mapping_epoch":13,"log_start":"141'332415","ondisk_log_start":"141'332415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3874980193,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":336986,"num_read_kb":1322570,"num_write":335659,"num_write_kb":1342446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"143'348801","reported_seq":683171,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:49.375886+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:33:49.375886+0000","last_peered":"2023-10-14T01:33:49.375886+0000","last_clean":"2023-10-14T01:33:49.375886+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:33:49.375886+0000","last_undegraded":"2023-10-14T01:33:49.375886+0000","last_fullsized":"2023-10-14T01:33:49.375886+0000","mapping_epoch":13,"log_start":"141'338717","ondisk_log_start":"141'338717","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3783557556,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":334313,"num_read_kb":1302939,"num_write":327529,"num_write_kb":1309759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":358,"reported_epoch":142,"state":"active+clean","last_fresh":"2023-10-14T01:31:50.742161+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:31:50.742161+0000","last_peered":"2023-10-14T01:31:50.742161+0000","last_clean":"2023-10-14T01:31:50.742161+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:31:50.742161+0000","last_undegraded":"2023-10-14T01:31:50.742161+0000","last_fullsized":"2023-10-14T01:31:50.742161+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29624953900,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2599446,"num_read_kb":10163746,"num_write":2567961,"num_write_kb":10269761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21032710144,"data_stored":30908794968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80477,"ondisk_log_size":80477,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739493,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5389364,"kb_used_data":4010988,"kb_used_omap":534,"kb_used_meta":1377833,"kb_avail":88982476,"statfs":{"total":96636764160,"available":91118055424,"internally_reserved":0,"allocated":4107251712,"data_stored":6035540189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1410901156},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,1,6,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.63},"min":{"1min":0.26700000000000002,"5min":0.193,"15min":0.182},"max":{"1min":4.9530000000000003,"5min":10.244,"15min":10.244},"last":0.317},{"interface":"front","average":{"1min":0.995,"5min":1.216,"15min":1.1279999999999999},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":3.7570000000000001,"5min":16.097999999999999,"15min":16.097999999999999},"last":0.34599999999999997}]},{"osd":1,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.8089999999999999,"15min":1.877},"min":{"1min":0.27600000000000002,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":11.196,"5min":20.43,"15min":44.308},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.77500000000000002,"15min":1.448},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.97099999999999997,"5min":10.471,"15min":44.350000000000001},"last":0.36799999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739481,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12325824,"kb_used_data":10270700,"kb_used_omap":816,"kb_used_meta":2054287,"kb_avail":82046016,"statfs":{"total":96636764160,"available":84015120384,"internally_reserved":0,"allocated":10517196800,"data_stored":15455056496,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":836331,"internal_metadata":2103590165},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,6,8,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.64000000000000001,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":0.67000000000000004,"5min":8.4469999999999992,"15min":16.481000000000002},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72799999999999998,"15min":0.89800000000000002},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":3.9390000000000001,"5min":8.407,"15min":16.52},"last":0.40899999999999997}]},{"osd":2,"last update":"Sat Oct 14 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.0569999999999999,"15min":0.98699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":5.4009999999999998,"5min":6.5789999999999997,"15min":10.375999999999999},"last":3.403},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.95099999999999996,"15min":0.88700000000000001},"min":{"1min":0.23899999999999999,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":5.0910000000000002,"5min":6.8369999999999997,"15min":10.308},"last":3.0830000000000002}]}]},{"osd":0,"up_from":8,"seq":34359739495,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7774952,"kb_used_data":6260320,"kb_used_omap":790,"kb_used_meta":1513833,"kb_avail":86596888,"statfs":{"total":96636764160,"available":88675213312,"internally_reserved":0,"allocated":6410567680,"data_stored":9419423159,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":1550165281},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.66300000000000003,"15min":1.1970000000000001},"min":{"1min":0.245,"5min":0.191,"15min":0.191},"max":{"1min":1.0509999999999999,"5min":5.5300000000000002,"15min":10.897},"last":0.22800000000000001},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.081,"15min":1.2589999999999999},"min":{"1min":0.24199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":14.532999999999999,"5min":14.532999999999999,"15min":14.532999999999999},"last":2.976}]},{"osd":2,"last update":"Sat Oct 14 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.59799999999999998,"15min":0.65400000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.185},"max":{"1min":0.73299999999999998,"5min":5.5949999999999998,"15min":7.1319999999999997},"last":0.443},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":0.872},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":4.3230000000000004,"5min":6.1040000000000001,"15min":7.6289999999999996},"last":0.39500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6410010624,"data_stored":9419048819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10516422656,"data_stored":15454465068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4106276864,"data_stored":6035281081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:33:52.077 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:33:52.077 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:33:52.078 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:33:52.827 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:52.822+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:52.846 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:52.842+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:52.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:52.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:52.883 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:52.878+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:52.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:52.878+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:52.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:52.878+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:52.894 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:52.886+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:53.219 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.214+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:33:53.265 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.262+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:53.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:53.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.278+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:53.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.278+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:53.283 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.278+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:53.284 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.278+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:53.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.954+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:54.003 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:53.998+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:54.143 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:54.137+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:54.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:54.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:54.158 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:54.153+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:54.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:54.153+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:54.159 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:54.153+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:54.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:54.161+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:57.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.769+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:33:57.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:57.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:57.779 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.773+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:57.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:57.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.777+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:57.786 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.777+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:57.787 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.777+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:57.790 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.781+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:57.943 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.937+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:57.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.957+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:57.978 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.973+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:33:57.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:57.985 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.977+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:57.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.977+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:57.986 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.977+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:33:57.991 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:57.981+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:33:58.242 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:58.251 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:58.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:58.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.249+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:58.311 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.305+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:33:58.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.305+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:33:58.322 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.317+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:58.333 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:58.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:58.347 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.341+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:58.545 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:33:58.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:33:58.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:33:58.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.601+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:33:58.608 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.601+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:33:58.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:33:58.621+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:00.268 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.265+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:34:00.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:00.288 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:00.334 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:00.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.365+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:00.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.365+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:00.374 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.365+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:00.376 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:00.373+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:00.641 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:00.637+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:34:00.672 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:00.669+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:00.680 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:00.677+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:02.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:02.124 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.121+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:02.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:02.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:02.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:02.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.133+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:02.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.133+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:02.187 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.181+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:02.259 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:02.314 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:02.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.301+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:02.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:02.332 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.329+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:02.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.337+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:02.342 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.337+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:02.360 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:02.357+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:03.853 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:03.849+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:03.874 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:03.869+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:03.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:03.877+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:03.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:03.881+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:03.884 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:03.881+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:03.885 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:03.881+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:03.904 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:03.901+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:04.227 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:04.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:04.253 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:04.245+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:04.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:04.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:04.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:04.253+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:04.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:04.253+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:04.264 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:04.253+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:04.981 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:04.977+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:34:05.018 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:05.013+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:05.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:05.453+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:05.512 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:05.501+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:05.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:05.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:05.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:05.505+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:05.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:05.505+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:05.515 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:05.505+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:05.556 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:05.549+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:05.763 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:05.757+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:08.076 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:08.073+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:34:08.122 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:08.117+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:34:08.128 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:08.125+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:08.144 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:08.141+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:09.010 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:09.005+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:09.987 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:09.981+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:34:10.036 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.029+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:34:10.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.037+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:34:10.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.037+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:10.060 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.057+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:10.076 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.069+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:10.078 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:10.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.073+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:10.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.073+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:10.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.073+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:10.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.085+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:10.964 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:10.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:10.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.953+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:10.966 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:10.953+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:11.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:11.721+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:11.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:11.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:11.746 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:11.741+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:12.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:34:12.912 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:34:12.913 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:34:12.968 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2979,"stamp":"2023-10-14T01:34:11.070348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29660098124,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2607918,"num_read_kb":10197171,"num_write":2576583,"num_write_kb":10306121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80667,"ondisk_log_size":80667,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25538788,"kb_used_data":20608900,"kb_used_omap":2142,"kb_used_meta":4927713,"kb_avail":257576732,"statfs":{"total":289910292480,"available":263758573568,"internally_reserved":0,"allocated":21103513600,"data_stored":30978507476,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2193702,"internal_metadata":5045978842},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,11,8,5,15,16,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17117184,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4191,"num_read_kb":16724,"num_write":4184,"num_write_kb":16736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001575"},"pg_stats":[{"pgid":"2.7","version":"144'302393","reported_seq":600835,"reported_epoch":144,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:34:09.940418+0000","last_change":"2023-10-14T01:26:34.487475+0000","last_active":"2023-10-14T01:34:09.940418+0000","last_peered":"2023-10-14T01:34:09.940418+0000","last_clean":"2023-10-14T01:34:09.940418+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:34:09.940418+0000","last_undegraded":"2023-10-14T01:34:09.940418+0000","last_fullsized":"2023-10-14T01:34:09.940418+0000","mapping_epoch":13,"log_start":"141'292313","ondisk_log_start":"141'292313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1589,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 455s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3450532023,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":299050,"num_read_kb":1180804,"num_write":300217,"num_write_kb":1200853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"144'318695","reported_seq":627966,"reported_epoch":144,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:34:09.927604+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:34:09.927604+0000","last_peered":"2023-10-14T01:34:09.927604+0000","last_clean":"2023-10-14T01:34:09.927604+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:34:09.927604+0000","last_undegraded":"2023-10-14T01:34:09.927604+0000","last_fullsized":"2023-10-14T01:34:09.927604+0000","mapping_epoch":13,"log_start":"141'308611","ondisk_log_start":"141'308611","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1392,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 460s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3481567499,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":310757,"num_read_kb":1205057,"num_write":302244,"num_write_kb":1208575,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"144'325268","reported_seq":645117,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:09.936702+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:34:09.936702+0000","last_peered":"2023-10-14T01:34:09.936702+0000","last_clean":"2023-10-14T01:34:09.936702+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:34:09.936702+0000","last_undegraded":"2023-10-14T01:34:09.936702+0000","last_fullsized":"2023-10-14T01:34:09.936702+0000","mapping_epoch":13,"log_start":"141'315214","ondisk_log_start":"141'315214","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3659866667,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":320299,"num_read_kb":1261014,"num_write":318131,"num_write_kb":1272403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"144'344167","reported_seq":675827,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:09.873608+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:34:09.873608+0000","last_peered":"2023-10-14T01:34:09.873608+0000","last_clean":"2023-10-14T01:34:09.873608+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:34:09.873608+0000","last_undegraded":"2023-10-14T01:34:09.873608+0000","last_fullsized":"2023-10-14T01:34:09.873608+0000","mapping_epoch":13,"log_start":"141'334113","ondisk_log_start":"141'334113","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3762123183,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":331918,"num_read_kb":1293855,"num_write":327566,"num_write_kb":1309946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"144'330091","reported_seq":659559,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:09.871971+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:34:09.871971+0000","last_peered":"2023-10-14T01:34:09.871971+0000","last_clean":"2023-10-14T01:34:09.871971+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:34:09.871971+0000","last_undegraded":"2023-10-14T01:34:09.871971+0000","last_fullsized":"2023-10-14T01:34:09.871971+0000","mapping_epoch":13,"log_start":"141'320012","ondisk_log_start":"141'320012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3704185303,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":328942,"num_read_kb":1274947,"num_write":322220,"num_write_kb":1288373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"144'347482","reported_seq":690649,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:09.115907+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:34:09.115907+0000","last_peered":"2023-10-14T01:34:09.115907+0000","last_clean":"2023-10-14T01:34:09.115907+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:34:09.115907+0000","last_undegraded":"2023-10-14T01:34:09.115907+0000","last_fullsized":"2023-10-14T01:34:09.115907+0000","mapping_epoch":13,"log_start":"141'337412","ondisk_log_start":"141'337412","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3933708708,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":343363,"num_read_kb":1347232,"num_write":340651,"num_write_kb":1362430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"144'343614","reported_seq":681482,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:09.079963+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:34:09.079963+0000","last_peered":"2023-10-14T01:34:09.079963+0000","last_clean":"2023-10-14T01:34:09.079963+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:34:09.079963+0000","last_undegraded":"2023-10-14T01:34:09.079963+0000","last_fullsized":"2023-10-14T01:34:09.079963+0000","mapping_epoch":13,"log_start":"141'333515","ondisk_log_start":"141'333515","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3879575905,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":338119,"num_read_kb":1327090,"num_write":336782,"num_write_kb":1346938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"144'349937","reported_seq":685354,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:09.939720+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:34:09.939720+0000","last_peered":"2023-10-14T01:34:09.939720+0000","last_clean":"2023-10-14T01:34:09.939720+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:34:09.939720+0000","last_undegraded":"2023-10-14T01:34:09.939720+0000","last_fullsized":"2023-10-14T01:34:09.939720+0000","mapping_epoch":13,"log_start":"141'339917","ondisk_log_start":"141'339917","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3787948468,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":335344,"num_read_kb":1307063,"num_write":328603,"num_write_kb":1314055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":360,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:54.453695+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:33:54.453695+0000","last_peered":"2023-10-14T01:33:54.453695+0000","last_clean":"2023-10-14T01:33:54.453695+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:33:54.453695+0000","last_undegraded":"2023-10-14T01:33:54.453695+0000","last_fullsized":"2023-10-14T01:33:54.453695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29659507756,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2607792,"num_read_kb":10197062,"num_write":2576414,"num_write_kb":10303573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21101883392,"data_stored":30977968216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80540,"ondisk_log_size":80540,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739497,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5475020,"kb_used_data":4024260,"kb_used_omap":534,"kb_used_meta":1450217,"kb_avail":88896820,"statfs":{"total":96636764160,"available":91030343680,"internally_reserved":0,"allocated":4120842240,"data_stored":6049130717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1485022372},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.63},"min":{"1min":0.26700000000000002,"5min":0.193,"15min":0.182},"max":{"1min":4.9530000000000003,"5min":10.244,"15min":10.244},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.995,"5min":1.216,"15min":1.1279999999999999},"min":{"1min":0.22900000000000001,"5min":0.19,"15min":0.16400000000000001},"max":{"1min":3.7570000000000001,"5min":16.097999999999999,"15min":16.097999999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Sat Oct 14 01:33:07 2023","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.8089999999999999,"15min":1.877},"min":{"1min":0.27600000000000002,"5min":0.223,"15min":0.20999999999999999},"max":{"1min":11.196,"5min":20.43,"15min":44.308},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.77500000000000002,"15min":1.448},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.97099999999999997,"5min":10.471,"15min":44.350000000000001},"last":0.46300000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739485,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12156228,"kb_used_data":10304184,"kb_used_omap":816,"kb_used_meta":1851215,"kb_avail":82215612,"statfs":{"total":96636764160,"available":84188786688,"internally_reserved":0,"allocated":10551484416,"data_stored":15489339224,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":836331,"internal_metadata":1895644437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,9,4,1,9,13,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.64000000000000001,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":0.67000000000000004,"5min":8.4469999999999992,"15min":16.481000000000002},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72799999999999998,"15min":0.89800000000000002},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":3.9390000000000001,"5min":8.407,"15min":16.52},"last":0.36899999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.0569999999999999,"15min":0.98699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":5.4009999999999998,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.95099999999999996,"15min":0.88700000000000001},"min":{"1min":0.23899999999999999,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":5.0910000000000002,"5min":6.8369999999999997,"15min":10.308},"last":0.23499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739499,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":7907540,"kb_used_data":6280456,"kb_used_omap":790,"kb_used_meta":1626281,"kb_avail":86464300,"statfs":{"total":96636764160,"available":88539443200,"internally_reserved":0,"allocated":6431186944,"data_stored":9440037535,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":1665312033},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.66300000000000003,"15min":1.1970000000000001},"min":{"1min":0.245,"5min":0.191,"15min":0.191},"max":{"1min":1.0509999999999999,"5min":5.5300000000000002,"15min":10.897},"last":0.26400000000000001},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.081,"15min":1.2589999999999999},"min":{"1min":0.24199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":14.532999999999999,"5min":14.532999999999999,"15min":14.532999999999999},"last":0.182}]},{"osd":2,"last update":"Sat Oct 14 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.59799999999999998,"15min":0.65400000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.185},"max":{"1min":0.73299999999999998,"5min":5.5949999999999998,"15min":7.1319999999999997},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":0.872},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":4.3230000000000004,"5min":6.1040000000000001,"15min":7.6289999999999996},"last":0.217}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6430769152,"data_stored":9439807347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10551246848,"data_stored":15489289260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4119867392,"data_stored":6048871609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:34:12.969 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:34:12.970 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:34:12.970 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:34:13.666 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:13.661+0000 7ff181d25640 -1 reset not still connected to 0x5628cc663860 2023-10-14T01:34:13.717 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:13.713+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:13.767 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:13.757+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:14.126 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.121+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:34:14.133 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:14.167 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.157+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:14.194 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.189+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:14.212 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:14.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.205+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:14.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.205+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:14.213 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.205+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:14.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.245+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:14.505 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:14.501+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:14.537 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:14.533+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:14.800 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.793+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:14.820 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.813+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:14.825 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.821+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:14.867 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:14.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.853+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:14.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.853+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:14.868 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.853+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:14.899 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.893+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:14.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.917+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:34:14.924 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.917+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1ad0 2023-10-14T01:34:14.925 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.921+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:34:14.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:14.926 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.921+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:14.941 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.933+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:14.948 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:14.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.945+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:14.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.945+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:14.949 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.945+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:14.965 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:14.961+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:15.166 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:15.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:15.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:15.197+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:15.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:15.201+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:15.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:15.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:15.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:15.217+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:15.221 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:15.217+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:15.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:15.229+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:16.286 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:16.281+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:16.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:16.302 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:16.297+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:16.303 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:16.297+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:16.312 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:16.305+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:17.560 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:17.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:17.630 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:17.625+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:17.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:17.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:17.637 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:17.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:17.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:17.629+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:17.638 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:17.629+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:17.653 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:17.649+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:19.098 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.093+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:19.105 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.101+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:19.113 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.109+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:19.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.125+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:19.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.133+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:19.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.133+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:19.139 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.133+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:19.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.161+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:19.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.225+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:19.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.229+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:19.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:19.324 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.321+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:19.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.321+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:19.325 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.321+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:19.373 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.369+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:19.626 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.621+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:19.639 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.629+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:19.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:19.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.649+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:19.654 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.649+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:19.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.649+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:19.662 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:19.657+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:20.550 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:20.545+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:20.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:20.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:20.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:20.581+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:20.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:20.581+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:20.584 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:20.581+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:21.330 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:21.325+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:22.427 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:22.421+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:22.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:22.441+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:22.478 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:22.469+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:24.065 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:24.061+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:24.069 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:24.065+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:24.090 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:24.077+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:24.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:24.085+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:24.091 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:24.085+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:24.962 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:24.953+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:24.993 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:24.985+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:25.007 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.001+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:25.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.001+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:25.008 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.001+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:25.179 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.173+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:25.188 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.185+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:25.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:25.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.221+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:25.232 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.221+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:25.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:25.221+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:26.530 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.525+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:34:26.543 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.537+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:26.546 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.541+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:26.553 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.549+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:26.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.553+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:26.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.569+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:26.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.569+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:26.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:26.577+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:27.813 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:27.809+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:27.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:27.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:27.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:27.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:27.851 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:27.845+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:27.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:27.845+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:27.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:27.845+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:27.852 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:27.845+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:29.600 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:29.593+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:29.640 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:29.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:29.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:29.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:29.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:29.633+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:29.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:29.633+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:29.641 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:29.633+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:30.281 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:30.273+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:30.297 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:30.293+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:30.319 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:30.313+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:30.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:30.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:30.340 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:30.333+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:30.341 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:30.333+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:30.401 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:30.397+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:31.408 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:31.405+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:31.958 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:31.953+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:34:31.959 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:31.953+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:31.959 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:31.953+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:32.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:32.117+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:32.164 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:32.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:32.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:32.161+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:32.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:32.161+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:32.165 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:32.161+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:32.972 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:34:33.916 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:34:33.917 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:34:33.959 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":2990,"stamp":"2023-10-14T01:34:33.074128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29694508620,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2616432,"num_read_kb":10231163,"num_write":2585099,"num_write_kb":10340185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80593,"ondisk_log_size":80593,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25850188,"kb_used_data":20675248,"kb_used_omap":2142,"kb_used_meta":5172769,"kb_avail":257265332,"statfs":{"total":289910292480,"available":263439699968,"internally_reserved":0,"allocated":21171453952,"data_stored":31046004668,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2193702,"internal_metadata":5296916186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,5,10,7,16,15],"upper_bound":256},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17358848,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4225,"num_read_kb":16844,"num_write":4331,"num_write_kb":17324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002019"},"pg_stats":[{"pgid":"2.7","version":"144'303419","reported_seq":602821,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:30.396047+0000","last_change":"2023-10-14T01:34:23.845114+0000","last_active":"2023-10-14T01:34:30.396047+0000","last_peered":"2023-10-14T01:34:30.396047+0000","last_clean":"2023-10-14T01:34:30.396047+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:34:30.396047+0000","last_undegraded":"2023-10-14T01:34:30.396047+0000","last_fullsized":"2023-10-14T01:34:30.396047+0000","mapping_epoch":13,"log_start":"141'293413","ondisk_log_start":"141'293413","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'303091","last_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_deep_scrub":"144'303091","last_deep_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_clean_scrub_stamp":"2023-10-14T01:34:23.844932+0000","objects_scrubbed":1656,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":470,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:45:42.906466+0000","scrub_duration":469.35733477000002,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3454464183,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":300000,"num_read_kb":1184600,"num_write":301192,"num_write_kb":1204753,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"144'319911","reported_seq":630266,"reported_epoch":144,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:34:30.388732+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:34:30.388732+0000","last_peered":"2023-10-14T01:34:30.388732+0000","last_clean":"2023-10-14T01:34:30.388732+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:34:30.388732+0000","last_undegraded":"2023-10-14T01:34:30.388732+0000","last_fullsized":"2023-10-14T01:34:30.388732+0000","mapping_epoch":13,"log_start":"141'309811","ondisk_log_start":"141'309811","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1482,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 480s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3485929739,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":311825,"num_read_kb":1209321,"num_write":303319,"num_write_kb":1212875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"144'326294","reported_seq":647120,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:30.375044+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:34:30.375044+0000","last_peered":"2023-10-14T01:34:30.375044+0000","last_clean":"2023-10-14T01:34:30.375044+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:34:30.375044+0000","last_undegraded":"2023-10-14T01:34:30.375044+0000","last_fullsized":"2023-10-14T01:34:30.375044+0000","mapping_epoch":13,"log_start":"141'316214","ondisk_log_start":"141'316214","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3663913515,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":321269,"num_read_kb":1264890,"num_write":319135,"num_write_kb":1276419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"144'345359","reported_seq":678139,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:30.364624+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:34:30.364624+0000","last_peered":"2023-10-14T01:34:30.364624+0000","last_clean":"2023-10-14T01:34:30.364624+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:34:30.364624+0000","last_undegraded":"2023-10-14T01:34:30.364624+0000","last_fullsized":"2023-10-14T01:34:30.364624+0000","mapping_epoch":13,"log_start":"141'335313","ondisk_log_start":"141'335313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3766690223,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":333026,"num_read_kb":1298283,"num_write":328691,"num_write_kb":1314446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"144'331143","reported_seq":661707,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:30.012365+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:34:30.012365+0000","last_peered":"2023-10-14T01:34:30.012365+0000","last_clean":"2023-10-14T01:34:30.012365+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:34:30.012365+0000","last_undegraded":"2023-10-14T01:34:30.012365+0000","last_fullsized":"2023-10-14T01:34:30.012365+0000","mapping_epoch":13,"log_start":"141'321112","ondisk_log_start":"141'321112","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3708330455,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":330018,"num_read_kb":1279243,"num_write":323252,"num_write_kb":1292501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"144'348528","reported_seq":692764,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:29.279337+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:34:29.279337+0000","last_peered":"2023-10-14T01:34:29.279337+0000","last_clean":"2023-10-14T01:34:29.279337+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:34:29.279337+0000","last_undegraded":"2023-10-14T01:34:29.279337+0000","last_fullsized":"2023-10-14T01:34:29.279337+0000","mapping_epoch":13,"log_start":"141'338512","ondisk_log_start":"141'338512","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3937927588,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":344423,"num_read_kb":1351460,"num_write":341687,"num_write_kb":1366574,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"144'344808","reported_seq":683861,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:29.230613+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:34:29.230613+0000","last_peered":"2023-10-14T01:34:29.230613+0000","last_clean":"2023-10-14T01:34:29.230613+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:34:29.230613+0000","last_undegraded":"2023-10-14T01:34:29.230613+0000","last_fullsized":"2023-10-14T01:34:29.230613+0000","mapping_epoch":13,"log_start":"141'334715","ondisk_log_start":"141'334715","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3884339553,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":339293,"num_read_kb":1331770,"num_write":337966,"num_write_kb":1351674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"144'351111","reported_seq":687649,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:30.388290+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:34:30.388290+0000","last_peered":"2023-10-14T01:34:30.388290+0000","last_clean":"2023-10-14T01:34:30.388290+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:34:30.388290+0000","last_undegraded":"2023-10-14T01:34:30.388290+0000","last_fullsized":"2023-10-14T01:34:30.388290+0000","mapping_epoch":13,"log_start":"141'341017","ondisk_log_start":"141'341017","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3792322996,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":336452,"num_read_kb":1311487,"num_write":329688,"num_write_kb":1318395,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":360,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:54.453695+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:33:54.453695+0000","last_peered":"2023-10-14T01:33:54.453695+0000","last_clean":"2023-10-14T01:33:54.453695+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:33:54.453695+0000","last_undegraded":"2023-10-14T01:33:54.453695+0000","last_fullsized":"2023-10-14T01:33:54.453695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29693918252,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2616306,"num_read_kb":10231054,"num_write":2584930,"num_write_kb":10337637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21170601984,"data_stored":31046195288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80466,"ondisk_log_size":80466,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739501,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5519900,"kb_used_data":4037204,"kb_used_omap":534,"kb_used_meta":1482153,"kb_avail":88851940,"statfs":{"total":96636764160,"available":90984386560,"internally_reserved":0,"allocated":4134096896,"data_stored":6062302661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1517724836},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:34:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":0.98399999999999999,"15min":0.68999999999999995},"min":{"1min":0.23200000000000001,"5min":0.193,"15min":0.182},"max":{"1min":8.8140000000000001,"5min":10.244,"15min":10.244},"last":0.42599999999999999},{"interface":"front","average":{"1min":1.413,"5min":1.3100000000000001,"15min":1.1859999999999999},"min":{"1min":0.216,"5min":0.193,"15min":0.16400000000000001},"max":{"1min":5.4390000000000001,"5min":16.097999999999999,"15min":16.097999999999999},"last":0.60399999999999998}]},{"osd":1,"last update":"Sat Oct 14 01:34:12 2023","interfaces":[{"interface":"back","average":{"1min":2.036,"5min":1.9419999999999999,"15min":1.786},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":8.7780000000000005,"5min":20.43,"15min":34.158000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":1.413,"5min":0.93200000000000005,"15min":1.3129999999999999},"min":{"1min":0.25700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":8.8539999999999992,"5min":10.471,"15min":34.192999999999998},"last":0.45500000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739489,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12316468,"kb_used_data":10337388,"kb_used_omap":816,"kb_used_meta":1978255,"kb_avail":82055372,"statfs":{"total":96636764160,"available":84024700928,"internally_reserved":0,"allocated":10585485312,"data_stored":15523118936,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":836331,"internal_metadata":2025733397},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,2,5,5,9,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:33:40 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.64000000000000001,"15min":0.79100000000000004},"min":{"1min":0.22800000000000001,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":0.67000000000000004,"5min":8.4469999999999992,"15min":16.481000000000002},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.72799999999999998,"15min":0.89800000000000002},"min":{"1min":0.26200000000000001,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":3.9390000000000001,"5min":8.407,"15min":16.52},"last":0.40400000000000003}]},{"osd":2,"last update":"Sat Oct 14 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.0569999999999999,"15min":0.98699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":5.4009999999999998,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.95099999999999996,"15min":0.88700000000000001},"min":{"1min":0.23899999999999999,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":5.0910000000000002,"5min":6.8369999999999997,"15min":10.308},"last":5.8040000000000003}]}]},{"osd":0,"up_from":8,"seq":34359739503,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8013820,"kb_used_data":6300656,"kb_used_omap":790,"kb_used_meta":1712361,"kb_avail":86358020,"statfs":{"total":96636764160,"available":88430612480,"internally_reserved":0,"allocated":6451871744,"data_stored":9460583071,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":1753457953},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.66300000000000003,"15min":1.1970000000000001},"min":{"1min":0.245,"5min":0.191,"15min":0.191},"max":{"1min":1.0509999999999999,"5min":5.5300000000000002,"15min":10.897},"last":0.42599999999999999},{"interface":"front","average":{"1min":2.1709999999999998,"5min":2.081,"15min":1.2589999999999999},"min":{"1min":0.24199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":14.532999999999999,"5min":14.532999999999999,"15min":14.532999999999999},"last":0.437}]},{"osd":2,"last update":"Sat Oct 14 01:33:37 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.59799999999999998,"15min":0.65400000000000003},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.185},"max":{"1min":0.73299999999999998,"5min":5.5949999999999998,"15min":7.1319999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.97399999999999998,"15min":0.872},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":4.3230000000000004,"5min":6.1040000000000001,"15min":7.6289999999999996},"last":0.33100000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6451548160,"data_stored":9460438899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10585878528,"data_stored":15523658796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4133175296,"data_stored":6062097593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:34:33.961 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:34:33.961 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:34:34.603 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:34.596+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:36.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:36.476+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:36.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:36.500+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:36.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:36.504+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:36.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:36.512+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:36.517 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:36.512+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:36.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:36.516+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:37.169 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:37.164+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:34:37.207 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:37.200+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:37.213 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:37.208+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:38.713 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:38.708+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:34:38.728 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:38.724+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:39.120 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.116+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:39.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:39.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:39.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.124+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:39.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.124+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:39.132 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.124+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:39.146 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.140+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:39.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.448+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:34:39.461 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.456+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:39.492 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:39.493 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.484+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:39.497 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:39.498 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:39.499 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:39.500 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.488+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:39.506 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:39.500+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:41.893 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:41.888+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:41.913 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:41.908+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:41.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:41.912+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:41.954 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:41.944+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:42.702 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:42.696+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:43.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:43.592+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:43.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:43.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:43.655 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:43.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:43.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:43.648+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:43.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:43.648+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:43.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:43.648+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:43.981 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:43.976+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:45.273 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:45.264+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:49.106 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:49.100+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:49.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:49.552+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:49.580 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:49.576+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:49.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:49.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:49.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:49.596+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:49.604 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:49.596+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:50.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.420+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:50.438 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:50.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.436+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:50.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.436+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:50.447 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.436+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:50.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:50.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:50.672 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:50.668+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:34:50.680 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:50.676+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:52.460 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:52.456+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:34:52.479 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:34:52.472+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:34:53.963 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:34:54.843 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:34:54.844 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:34:54.889 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":3000,"stamp":"2023-10-14T01:34:53.081357+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29730033228,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2625315,"num_read_kb":10266611,"num_write":2594085,"num_write_kb":10376129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80573,"ondisk_log_size":80573,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":24967456,"kb_used_data":20745796,"kb_used_omap":2160,"kb_used_meta":4219471,"kb_avail":258148064,"statfs":{"total":289910292480,"available":264343617536,"internally_reserved":0,"allocated":21243695104,"data_stored":31116980156,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2212675,"internal_metadata":4320738493},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,6,6,24,21,11,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17752064,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4376,"num_read_kb":17460,"num_write":4490,"num_write_kb":17960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001970"},"pg_stats":[{"pgid":"2.7","version":"144'304394","reported_seq":604716,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:50.798507+0000","last_change":"2023-10-14T01:34:23.845114+0000","last_active":"2023-10-14T01:34:50.798507+0000","last_peered":"2023-10-14T01:34:50.798507+0000","last_clean":"2023-10-14T01:34:50.798507+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:34:50.798507+0000","last_undegraded":"2023-10-14T01:34:50.798507+0000","last_fullsized":"2023-10-14T01:34:50.798507+0000","mapping_epoch":13,"log_start":"141'294313","ondisk_log_start":"141'294313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'303091","last_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_deep_scrub":"144'303091","last_deep_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_clean_scrub_stamp":"2023-10-14T01:34:23.844932+0000","objects_scrubbed":1656,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":470,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:45:42.906466+0000","scrub_duration":469.35733477000002,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3458265271,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":300921,"num_read_kb":1188268,"num_write":302167,"num_write_kb":1208653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"144'321142","reported_seq":632659,"reported_epoch":144,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:34:50.800926+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:34:50.800926+0000","last_peered":"2023-10-14T01:34:50.800926+0000","last_clean":"2023-10-14T01:34:50.800926+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:34:50.800926+0000","last_undegraded":"2023-10-14T01:34:50.800926+0000","last_fullsized":"2023-10-14T01:34:50.800926+0000","mapping_epoch":13,"log_start":"142'311111","ondisk_log_start":"142'311111","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1530,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 501s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3490513163,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":312965,"num_read_kb":1213881,"num_write":304489,"num_write_kb":1217555,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"144'327376","reported_seq":649330,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:50.780475+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:34:50.780475+0000","last_peered":"2023-10-14T01:34:50.780475+0000","last_clean":"2023-10-14T01:34:50.780475+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:34:50.780475+0000","last_undegraded":"2023-10-14T01:34:50.780475+0000","last_fullsized":"2023-10-14T01:34:50.780475+0000","mapping_epoch":13,"log_start":"141'317314","ondisk_log_start":"141'317314","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3668169259,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":322394,"num_read_kb":1269378,"num_write":320209,"num_write_kb":1280715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"144'346662","reported_seq":680636,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:50.782004+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:34:50.782004+0000","last_peered":"2023-10-14T01:34:50.782004+0000","last_clean":"2023-10-14T01:34:50.782004+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:34:50.782004+0000","last_undegraded":"2023-10-14T01:34:50.782004+0000","last_fullsized":"2023-10-14T01:34:50.782004+0000","mapping_epoch":13,"log_start":"142'336613","ondisk_log_start":"142'336613","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3771777455,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":334209,"num_read_kb":1303003,"num_write":329970,"num_write_kb":1319562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"144'332233","reported_seq":663902,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:50.136921+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:34:50.136921+0000","last_peered":"2023-10-14T01:34:50.136921+0000","last_clean":"2023-10-14T01:34:50.136921+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:34:50.136921+0000","last_undegraded":"2023-10-14T01:34:50.136921+0000","last_fullsized":"2023-10-14T01:34:50.136921+0000","mapping_epoch":13,"log_start":"141'322212","ondisk_log_start":"141'322212","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3712561623,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":331107,"num_read_kb":1283579,"num_write":324317,"num_write_kb":1296761,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"144'349841","reported_seq":695339,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:49.412833+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:34:49.412833+0000","last_peered":"2023-10-14T01:34:49.412833+0000","last_clean":"2023-10-14T01:34:49.412833+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:34:49.412833+0000","last_undegraded":"2023-10-14T01:34:49.412833+0000","last_fullsized":"2023-10-14T01:34:49.412833+0000","mapping_epoch":13,"log_start":"142'339812","ondisk_log_start":"142'339812","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3943076260,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":345681,"num_read_kb":1356492,"num_write":342978,"num_write_kb":1371738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"144'345893","reported_seq":686042,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:49.385421+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:34:49.385421+0000","last_peered":"2023-10-14T01:34:49.385421+0000","last_clean":"2023-10-14T01:34:49.385421+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:34:49.385421+0000","last_undegraded":"2023-10-14T01:34:49.385421+0000","last_fullsized":"2023-10-14T01:34:49.385421+0000","mapping_epoch":13,"log_start":"142'335815","ondisk_log_start":"142'335815","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3888525665,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":340383,"num_read_kb":1336130,"num_write":339029,"num_write_kb":1355926,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"144'352212","reported_seq":689847,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:50.792514+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:34:50.792514+0000","last_peered":"2023-10-14T01:34:50.792514+0000","last_clean":"2023-10-14T01:34:50.792514+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:34:50.792514+0000","last_undegraded":"2023-10-14T01:34:50.792514+0000","last_fullsized":"2023-10-14T01:34:50.792514+0000","mapping_epoch":13,"log_start":"142'342117","ondisk_log_start":"142'342117","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3796554164,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":337529,"num_read_kb":1315771,"num_write":330757,"num_write_kb":1322671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":360,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:54.453695+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:33:54.453695+0000","last_peered":"2023-10-14T01:33:54.453695+0000","last_clean":"2023-10-14T01:33:54.453695+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:33:54.453695+0000","last_undegraded":"2023-10-14T01:33:54.453695+0000","last_fullsized":"2023-10-14T01:33:54.453695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29729442860,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2625189,"num_read_kb":10266502,"num_write":2593916,"num_write_kb":10373581,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21241909248,"data_stored":31116249176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80446,"ondisk_log_size":80446,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739505,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5605012,"kb_used_data":4050188,"kb_used_omap":534,"kb_used_meta":1554281,"kb_avail":88766828,"statfs":{"total":96636764160,"available":90897231872,"internally_reserved":0,"allocated":4147392512,"data_stored":6075377093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1591583908},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,4,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:34:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":0.98399999999999999,"15min":0.68999999999999995},"min":{"1min":0.23200000000000001,"5min":0.193,"15min":0.182},"max":{"1min":8.8140000000000001,"5min":10.244,"15min":10.244},"last":0.224},{"interface":"front","average":{"1min":1.413,"5min":1.3100000000000001,"15min":1.1859999999999999},"min":{"1min":0.216,"5min":0.193,"15min":0.16400000000000001},"max":{"1min":5.4390000000000001,"5min":16.097999999999999,"15min":16.097999999999999},"last":0.27400000000000002}]},{"osd":1,"last update":"Sat Oct 14 01:34:12 2023","interfaces":[{"interface":"back","average":{"1min":2.036,"5min":1.9419999999999999,"15min":1.786},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":8.7780000000000005,"5min":20.43,"15min":34.158000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":1.413,"5min":0.93200000000000005,"15min":1.3129999999999999},"min":{"1min":0.25700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":8.8539999999999992,"5min":10.471,"15min":34.192999999999998},"last":0.45900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359739493,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11224408,"kb_used_data":10373584,"kb_used_omap":835,"kb_used_meta":849980,"kb_avail":83147432,"statfs":{"total":96636764160,"available":85142970368,"internally_reserved":0,"allocated":10622550016,"data_stored":15559528280,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":855304,"internal_metadata":870380280},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,4,18,13,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.52200000000000002,"15min":0.78500000000000003},"min":{"1min":0.223,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":1.6120000000000001,"5min":8.4469999999999992,"15min":16.481000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.76400000000000001,"15min":0.89000000000000001},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":3.331,"5min":8.407,"15min":16.52},"last":0.29699999999999999}]},{"osd":2,"last update":"Sat Oct 14 01:33:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.0569999999999999,"15min":0.98699999999999999},"min":{"1min":0.30199999999999999,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":5.4009999999999998,"5min":6.5789999999999997,"15min":10.375999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.95099999999999996,"15min":0.88700000000000001},"min":{"1min":0.23899999999999999,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":5.0910000000000002,"5min":6.8369999999999997,"15min":10.308},"last":1.996}]}]},{"osd":0,"up_from":8,"seq":34359739507,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8138036,"kb_used_data":6322024,"kb_used_omap":790,"kb_used_meta":1815209,"kb_avail":86233804,"statfs":{"total":96636764160,"available":88303415296,"internally_reserved":0,"allocated":6473752576,"data_stored":9482074783,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":1858774305},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:34:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.64000000000000001,"15min":1.143},"min":{"1min":0.217,"5min":0.19400000000000001,"15min":0.191},"max":{"1min":4.391,"5min":5.3760000000000003,"15min":10.897},"last":0.31900000000000001},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.7769999999999999,"15min":1.2909999999999999},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":3.5779999999999998,"5min":14.532999999999999,"15min":14.532999999999999},"last":0.45000000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:34:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.254,"5min":0.20499999999999999,"15min":0.185},"max":{"1min":4.4400000000000004,"5min":5.4580000000000002,"15min":7.1319999999999997},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.90500000000000003,"15min":0.873},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":2.7770000000000001,"5min":6.1040000000000001,"15min":7.6289999999999996},"last":0.23999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6473355264,"data_stored":9481840499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10621792256,"data_stored":15558954028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4146761728,"data_stored":6075454649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:34:54.890 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:34:54.891 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:34:55.295 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.288+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:55.305 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.296+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:55.315 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.308+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:55.327 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:55.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.320+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:55.328 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.320+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:55.335 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.332+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:55.370 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.364+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:55.400 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.396+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:55.443 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:55.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.428+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:55.445 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.428+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:55.446 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.428+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:55.448 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.444+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:55.449 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.444+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:34:55.482 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.468+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:55.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.480+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:55.496 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.488+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:55.523 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:55.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.516+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:55.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.516+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:55.524 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.516+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:55.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:55.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:56.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:56.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:56.211 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:56.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:56.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:56.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:56.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:56.208+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:56.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:56.208+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:56.217 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:56.208+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:57.154 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.148+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:57.207 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.200+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:57.233 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:57.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:57.234 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.228+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:57.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.228+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:57.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.228+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:34:57.235 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.228+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:57.609 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.600+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:57.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.624+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:57.636 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.632+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:57.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:57.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.652+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:57.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:57.684+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:34:58.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:58.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:34:58.106 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:58.096+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:34:58.114 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:58.108+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:34:58.136 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:58.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:34:58.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:58.128+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:34:58.137 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:58.128+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:34:58.155 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:34:58.148+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:00.508 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.492+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:35:00.518 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.512+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:35:00.564 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.560+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:00.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:00.581 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:00.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:00.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.584+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:00.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.584+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:00.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.584+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:00.589 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:00.584+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:01.298 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.292+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:01.391 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.384+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:01.667 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.660+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:01.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:01.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.696+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:01.703 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.696+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:01.704 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.696+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:01.736 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.732+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:01.764 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.756+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:01.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:01.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:01.764+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:02.216 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.212+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:02.241 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.236+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:02.250 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.244+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:02.252 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.248+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:02.276 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.268+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:02.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.268+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:02.277 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.268+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:02.280 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.276+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:02.953 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:02.944+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:35:02.955 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:02.948+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:02.978 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:02.972+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:03.001 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:02.996+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:03.011 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:03.004+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:03.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:03.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:03.017 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:03.012+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:03.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:03.012+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:03.018 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:03.012+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:03.026 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:03.020+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:04.282 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:04.280+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:04.336 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:04.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:04.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:04.324+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:04.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:04.324+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:04.337 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:04.324+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:04.362 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:04.360+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:05.081 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:05.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:05.086 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:05.080+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:05.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:05.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:05.102 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:05.100+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:05.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:05.100+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:05.103 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:05.100+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:05.418 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:05.416+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:35:05.482 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:05.480+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:05.688 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:05.684+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:08.669 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.668+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:35:08.715 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.712+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:08.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.736+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:08.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.748+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:08.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:08.773 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.772+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:08.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.772+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:08.774 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.772+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:08.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.792+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:08.801 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.800+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:35:08.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:08.822 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.816+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:08.836 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.832+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:08.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:08.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.836+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:08.839 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.836+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:08.859 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:08.856+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:10.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:10.344+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:10.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:10.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:10.383 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:10.380+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:10.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:10.380+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:10.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:10.380+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:10.528 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:10.528+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:35:10.557 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:10.548+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:10.557 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:10.548+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:11.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:11.061 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.060+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:11.071 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.068+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:11.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:11.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.076+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:11.079 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.076+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:11.080 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.076+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:11.083 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.080+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:11.566 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.564+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:11.576 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.572+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:11.618 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.616+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:11.633 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.632+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:35:11.645 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.644+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:11.672 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.668+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:11.673 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.672+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:11.686 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.684+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:11.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.692+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:11.961 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.960+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:11.979 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:11.976+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:12.138 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:12.136+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:12.184 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:12.184+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:12.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:12.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:12.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:12.192+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:12.199 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:12.192+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:12.200 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:12.192+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:12.202 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:12.196+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:14.892 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:35:14.994 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:14.991+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:15.006 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.003+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:15.028 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:15.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.027+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:15.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.027+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:15.029 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.027+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:15.034 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.027+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:15.439 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.431+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:15.464 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:15.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:15.468 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.467+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:15.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:15.511+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:15.684 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:35:15.685 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:35:15.753 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":3011,"stamp":"2023-10-14T01:35:15.091317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29767134796,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2634514,"num_read_kb":10303319,"num_write":2603573,"num_write_kb":10414081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80484,"ondisk_log_size":80484,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25225940,"kb_used_data":20816700,"kb_used_omap":2160,"kb_used_meta":4407055,"kb_avail":257889580,"statfs":{"total":289910292480,"available":264078929920,"internally_reserved":0,"allocated":21316300800,"data_stored":31187861436,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2212675,"internal_metadata":4512824509},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,1,11,10,22,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19603456,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4911,"num_read_kb":19612,"num_write":5040,"num_write_kb":20160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002008"},"pg_stats":[{"pgid":"2.7","version":"144'305380","reported_seq":606719,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:35:11.380458+0000","last_change":"2023-10-14T01:34:23.845114+0000","last_active":"2023-10-14T01:35:11.380458+0000","last_peered":"2023-10-14T01:35:11.380458+0000","last_clean":"2023-10-14T01:35:11.380458+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:35:11.380458+0000","last_undegraded":"2023-10-14T01:35:11.380458+0000","last_fullsized":"2023-10-14T01:35:11.380458+0000","mapping_epoch":13,"log_start":"141'295313","ondisk_log_start":"141'295313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'303091","last_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_deep_scrub":"144'303091","last_deep_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_clean_scrub_stamp":"2023-10-14T01:34:23.844932+0000","objects_scrubbed":1656,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":470,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:45:42.906466+0000","scrub_duration":469.35733477000002,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3462078647,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":301937,"num_read_kb":1192324,"num_write":303153,"num_write_kb":1212597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"144'322330","reported_seq":634887,"reported_epoch":144,"state":"active+clean+scrubbing+deep","last_fresh":"2023-10-14T01:35:11.292779+0000","last_change":"2023-10-14T01:26:29.526948+0000","last_active":"2023-10-14T01:35:11.292779+0000","last_peered":"2023-10-14T01:35:11.292779+0000","last_clean":"2023-10-14T01:35:11.292779+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:35:11.292779+0000","last_undegraded":"2023-10-14T01:35:11.292779+0000","last_fullsized":"2023-10-14T01:35:11.292779+0000","mapping_epoch":13,"log_start":"142'312311","ondisk_log_start":"142'312311","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-13T23:59:42.774992+0000","last_clean_scrub_stamp":"2023-10-13T23:59:42.774992+0000","objects_scrubbed":1609,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 521s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3494658315,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":313988,"num_read_kb":1217965,"num_write":305559,"num_write_kb":1221835,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"144'328465","reported_seq":651474,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:35:11.295893+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:35:11.295893+0000","last_peered":"2023-10-14T01:35:11.295893+0000","last_clean":"2023-10-14T01:35:11.295893+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:35:11.295893+0000","last_undegraded":"2023-10-14T01:35:11.295893+0000","last_fullsized":"2023-10-14T01:35:11.295893+0000","mapping_epoch":13,"log_start":"142'318414","ondisk_log_start":"142'318414","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3672261163,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":323444,"num_read_kb":1273566,"num_write":321278,"num_write_kb":1284991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"144'347648","reported_seq":682578,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:35:11.257200+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:35:11.257200+0000","last_peered":"2023-10-14T01:35:11.257200+0000","last_clean":"2023-10-14T01:35:11.257200+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:35:11.257200+0000","last_undegraded":"2023-10-14T01:35:11.257200+0000","last_fullsized":"2023-10-14T01:35:11.257200+0000","mapping_epoch":13,"log_start":"142'337613","ondisk_log_start":"142'337613","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3775418799,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":335151,"num_read_kb":1306763,"num_write":330898,"num_write_kb":1323274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"144'333351","reported_seq":666152,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:35:10.312529+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:35:10.312529+0000","last_peered":"2023-10-14T01:35:10.312529+0000","last_clean":"2023-10-14T01:35:10.312529+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:35:10.312529+0000","last_undegraded":"2023-10-14T01:35:10.312529+0000","last_fullsized":"2023-10-14T01:35:10.312529+0000","mapping_epoch":13,"log_start":"142'323312","ondisk_log_start":"142'323312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3716899287,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":332216,"num_read_kb":1288011,"num_write":325425,"num_write_kb":1301193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"144'351621","reported_seq":698788,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:35:14.681846+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:35:14.681846+0000","last_peered":"2023-10-14T01:35:14.681846+0000","last_clean":"2023-10-14T01:35:14.681846+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:35:14.681846+0000","last_undegraded":"2023-10-14T01:35:14.681846+0000","last_fullsized":"2023-10-14T01:35:14.681846+0000","mapping_epoch":13,"log_start":"142'341612","ondisk_log_start":"142'341612","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3950055844,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":347340,"num_read_kb":1363100,"num_write":344740,"num_write_kb":1378786,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"144'347280","reported_seq":688722,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:35:14.662104+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:35:14.662104+0000","last_peered":"2023-10-14T01:35:14.662104+0000","last_clean":"2023-10-14T01:35:14.662104+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:35:14.662104+0000","last_undegraded":"2023-10-14T01:35:14.662104+0000","last_fullsized":"2023-10-14T01:35:14.662104+0000","mapping_epoch":13,"log_start":"142'337215","ondisk_log_start":"142'337215","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3893883233,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":341667,"num_read_kb":1341258,"num_write":340398,"num_write_kb":1361402,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"144'353489","reported_seq":692249,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:35:11.268511+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:35:11.268511+0000","last_peered":"2023-10-14T01:35:11.268511+0000","last_clean":"2023-10-14T01:35:11.268511+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:35:11.268511+0000","last_undegraded":"2023-10-14T01:35:11.268511+0000","last_fullsized":"2023-10-14T01:35:11.268511+0000","mapping_epoch":13,"log_start":"142'343417","ondisk_log_start":"142'343417","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3801289140,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":338645,"num_read_kb":1320223,"num_write":331953,"num_write_kb":1327455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":360,"reported_epoch":143,"state":"active+clean","last_fresh":"2023-10-14T01:33:54.453695+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:33:54.453695+0000","last_peered":"2023-10-14T01:33:54.453695+0000","last_clean":"2023-10-14T01:33:54.453695+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:33:54.453695+0000","last_undegraded":"2023-10-14T01:33:54.453695+0000","last_fullsized":"2023-10-14T01:33:54.453695+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29766544428,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2634388,"num_read_kb":10303210,"num_write":2603404,"num_write_kb":10411533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":13129,"num_omap_keys":242,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21315153920,"data_stored":31187712088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80357,"ondisk_log_size":80357,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739510,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5713396,"kb_used_data":4068332,"kb_used_omap":534,"kb_used_meta":1644521,"kb_avail":88658444,"statfs":{"total":96636764160,"available":90786246656,"internally_reserved":0,"allocated":4165971968,"data_stored":6093551045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1683989668},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,1,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:34:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4159999999999999,"5min":0.98399999999999999,"15min":0.68999999999999995},"min":{"1min":0.23200000000000001,"5min":0.193,"15min":0.182},"max":{"1min":8.8140000000000001,"5min":10.244,"15min":10.244},"last":0.502},{"interface":"front","average":{"1min":1.413,"5min":1.3100000000000001,"15min":1.1859999999999999},"min":{"1min":0.216,"5min":0.193,"15min":0.16400000000000001},"max":{"1min":5.4390000000000001,"5min":16.097999999999999,"15min":16.097999999999999},"last":3.3599999999999999}]},{"osd":1,"last update":"Sat Oct 14 01:34:12 2023","interfaces":[{"interface":"back","average":{"1min":2.036,"5min":1.9419999999999999,"15min":1.786},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":8.7780000000000005,"5min":20.43,"15min":34.158000000000001},"last":3.6779999999999999},{"interface":"front","average":{"1min":1.413,"5min":0.93200000000000005,"15min":1.3129999999999999},"min":{"1min":0.25700000000000001,"5min":0.214,"15min":0.214},"max":{"1min":8.8539999999999992,"5min":10.471,"15min":34.192999999999998},"last":0.53800000000000003}]}]},{"osd":1,"up_from":8,"seq":34359739497,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11278832,"kb_used_data":10407016,"kb_used_omap":835,"kb_used_meta":870972,"kb_avail":83093008,"statfs":{"total":96636764160,"available":85087240192,"internally_reserved":0,"allocated":10656784384,"data_stored":15592939352,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":855304,"internal_metadata":891876088},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,10,7,12,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.52200000000000002,"15min":0.78500000000000003},"min":{"1min":0.223,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":1.6120000000000001,"5min":8.4469999999999992,"15min":16.481000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.76400000000000001,"15min":0.89000000000000001},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":3.331,"5min":8.407,"15min":16.52},"last":0.45700000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.927,"5min":1.2390000000000001,"15min":1.016},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":30.763999999999999,"5min":30.763999999999999,"15min":30.763999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":2.2250000000000001,"5min":1.0860000000000001,"15min":0.98199999999999998},"min":{"1min":0.23499999999999999,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":26.440999999999999,"5min":26.440999999999999,"15min":26.440999999999999},"last":0.59999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359739511,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8233712,"kb_used_data":6341352,"kb_used_omap":790,"kb_used_meta":1891561,"kb_avail":86138128,"statfs":{"total":96636764160,"available":88205443072,"internally_reserved":0,"allocated":6493544448,"data_stored":9501371039,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":1936958753},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:34:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.64000000000000001,"15min":1.143},"min":{"1min":0.217,"5min":0.19400000000000001,"15min":0.191},"max":{"1min":4.391,"5min":5.3760000000000003,"15min":10.897},"last":0.17299999999999999},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.7769999999999999,"15min":1.2909999999999999},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":3.5779999999999998,"5min":14.532999999999999,"15min":14.532999999999999},"last":0.39300000000000002}]},{"osd":2,"last update":"Sat Oct 14 01:34:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.254,"5min":0.20499999999999999,"15min":0.185},"max":{"1min":4.4400000000000004,"5min":5.4580000000000002,"15min":7.1319999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.90500000000000003,"15min":0.873},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":2.7770000000000001,"5min":6.1040000000000001,"15min":7.6289999999999996},"last":0.373}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6493171712,"data_stored":9501144947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10657001472,"data_stored":15593294892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4164980736,"data_stored":6093272249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:35:15.754 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-13T23:59:42.774992+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=13, tm_hour=23, tm_min=59, tm_sec=42, tm_wday=4, tm_yday=286, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=14, tm_hour=1, tm_min=5, tm_sec=27, tm_wday=5, tm_yday=287, tm_isdst=0) 2023-10-14T01:35:15.755 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-14T01:35:16.686 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:16.683+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:16.689 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:16.683+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:16.957 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:16.955+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:16.997 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:16.995+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:17.656 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:17.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:17.692 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:17.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:17.706 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:17.703+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:17.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:17.711+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:17.717 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:17.711+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:17.737 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:17.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:17.778 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:17.775+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:18.114 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:18.111+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:20.616 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.615+0000 7faeed24b640 -1 reset not still connected to 0x556f2fbb1790 2023-10-14T01:35:20.657 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.655+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:20.670 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.667+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:20.689 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.687+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:20.697 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.695+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:20.729 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.727+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:20.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.727+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:20.730 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.727+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:20.738 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:20.731+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:22.978 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:22.975+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:35:23.014 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:23.011+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:24.388 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.383+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:24.428 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:24.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.423+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:24.429 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.423+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:24.430 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.423+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:24.958 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:24.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:24.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:24.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.955+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:24.959 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.955+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:24.960 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:24.955+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:25.695 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.691+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:35:25.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:25.718 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.711+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:25.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:25.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:25.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.751+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:25.754 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.751+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:25.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.751+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:25.755 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.751+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:25.772 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.771+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:25.795 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:25.815 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:25.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.811+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:25.816 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.811+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:25.817 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.811+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:25.849 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.847+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:25.866 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.863+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:25.914 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:25.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.911+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:25.915 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.911+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:25.916 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:25.911+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:26.437 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.435+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:26.453 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.451+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:26.465 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.463+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:26.487 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.483+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:26.503 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:26.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.499+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:26.504 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.499+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:26.519 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.515+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:26.603 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.599+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:26.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:26.628 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.623+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:26.634 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:26.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.631+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:26.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.631+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:26.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.631+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:26.635 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:26.631+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:26.705 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:26.703+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:27.240 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:27.239+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:35:27.257 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:27.255+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:27.262 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:27.259+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:30.366 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:30.363+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:30.882 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:30.879+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:30.924 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:30.923+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:31.781 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:31.775+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:31.873 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:31.871+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:32.501 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.499+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:32.514 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.511+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:32.529 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.527+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:32.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:32.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.539+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:32.541 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.539+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:32.542 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.539+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:32.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.747+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:35:32.753 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.747+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:32.769 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:32.770 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.767+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:32.793 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:32.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.791+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:32.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.791+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:32.794 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.791+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:32.837 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:32.831+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:34.112 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.107+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:34.122 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.119+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:34.131 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.127+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:34.208 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:34.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.203+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:34.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.203+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:34.209 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.203+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:34.238 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:34.235+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:35.042 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.035+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:35:35.043 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.035+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:35.052 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:35.053 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.051+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:35.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:35.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.067+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:35.070 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.067+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:35.092 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:35.091+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:35.756 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-14T01:35:36.531 INFO:teuthology.orchestra.run.smithi183.stdout: 2023-10-14T01:35:36.532 INFO:teuthology.orchestra.run.smithi183.stderr:dumped all 2023-10-14T01:35:36.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.555+0000 7faeed24b640 -1 reset not still connected to 0x556f313321a0 2023-10-14T01:35:36.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe4e0 2023-10-14T01:35:36.570 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.555+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbe680 2023-10-14T01:35:36.572 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.571+0000 7faeed24b640 -1 reset not still connected to 0x556f31cbf5f0 2023-10-14T01:35:36.583 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:36.584 INFO:teuthology.orchestra.run.smithi183.stdout:{"pg_ready":true,"pg_map":{"version":3022,"stamp":"2023-10-14T01:35:35.094725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29801643596,"num_objects":14181,"num_object_clones":8960,"num_object_copies":28362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14181,"num_whiteouts":0,"num_read":2643356,"num_read_kb":10338655,"num_write":2612648,"num_write_kb":10450381,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16715,"num_omap_keys":318,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80552,"ondisk_log_size":80552,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":25588052,"kb_used_data":20885624,"kb_used_omap":2160,"kb_used_meta":4700239,"kb_avail":257527468,"statfs":{"total":289910292480,"available":263708127232,"internally_reserved":0,"allocated":21386878976,"data_stored":31255356915,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":2212675,"internal_metadata":4813044925},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,14,13,9,7,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19595264,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5185,"num_read_kb":20724,"num_write":5193,"num_write_kb":20772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3586,"num_omap_keys":76,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002009"},"pg_stats":[{"pgid":"2.7","version":"145'306365","reported_seq":608685,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:31.745272+0000","last_change":"2023-10-14T01:34:23.845114+0000","last_active":"2023-10-14T01:35:31.745272+0000","last_peered":"2023-10-14T01:35:31.745272+0000","last_clean":"2023-10-14T01:35:31.745272+0000","last_became_active":"2023-10-13T23:59:43.806615+0000","last_became_peered":"2023-10-13T23:59:43.806615+0000","last_unstale":"2023-10-14T01:35:31.745272+0000","last_undegraded":"2023-10-14T01:35:31.745272+0000","last_fullsized":"2023-10-14T01:35:31.745272+0000","mapping_epoch":13,"log_start":"142'296313","ondisk_log_start":"142'296313","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"144'303091","last_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_deep_scrub":"144'303091","last_deep_scrub_stamp":"2023-10-14T01:34:23.844932+0000","last_clean_scrub_stamp":"2023-10-14T01:34:23.844932+0000","objects_scrubbed":1656,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":470,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T08:45:42.906466+0000","scrub_duration":469.35733477000002,"objects_trimmed":0,"snaptrim_duration":0.017930015000000001,"stat_sum":{"num_bytes":3465736375,"num_objects":1656,"num_object_clones":1056,"num_object_copies":3312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1656,"num_whiteouts":0,"num_read":302917,"num_read_kb":1196240,"num_write":304138,"num_write_kb":1216537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"145'323548","reported_seq":637286,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:31.761669+0000","last_change":"2023-10-14T01:35:25.215667+0000","last_active":"2023-10-14T01:35:31.761669+0000","last_peered":"2023-10-14T01:35:31.761669+0000","last_clean":"2023-10-14T01:35:31.761669+0000","last_became_active":"2023-10-13T23:59:43.806973+0000","last_became_peered":"2023-10-13T23:59:43.806973+0000","last_unstale":"2023-10-14T01:35:31.761669+0000","last_undegraded":"2023-10-14T01:35:31.761669+0000","last_fullsized":"2023-10-14T01:35:31.761669+0000","mapping_epoch":13,"log_start":"142'313511","ondisk_log_start":"142'313511","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"145'323160","last_scrub_stamp":"2023-10-14T01:35:25.215491+0000","last_deep_scrub":"145'323160","last_deep_scrub_stamp":"2023-10-14T01:35:25.215491+0000","last_clean_scrub_stamp":"2023-10-14T01:35:25.215491+0000","objects_scrubbed":1663,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":536,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T10:21:08.510094+0000","scrub_duration":535.68842134299996,"objects_trimmed":0,"snaptrim_duration":0.017401204999999999,"stat_sum":{"num_bytes":3498991883,"num_objects":1663,"num_object_clones":1051,"num_object_copies":3326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1663,"num_whiteouts":0,"num_read":315143,"num_read_kb":1222581,"num_write":306685,"num_write_kb":1226339,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3586,"num_omap_keys":76,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"145'329603","reported_seq":653666,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:31.734697+0000","last_change":"2023-10-14T01:26:30.348254+0000","last_active":"2023-10-14T01:35:31.734697+0000","last_peered":"2023-10-14T01:35:31.734697+0000","last_clean":"2023-10-14T01:35:31.734697+0000","last_became_active":"2023-10-13T23:59:43.807136+0000","last_became_peered":"2023-10-13T23:59:43.807136+0000","last_unstale":"2023-10-14T01:35:31.734697+0000","last_undegraded":"2023-10-14T01:35:31.734697+0000","last_fullsized":"2023-10-14T01:35:31.734697+0000","mapping_epoch":13,"log_start":"142'319514","ondisk_log_start":"142'319514","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'301997","last_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_deep_scrub":"137'301997","last_deep_scrub_stamp":"2023-10-14T01:26:30.348012+0000","last_clean_scrub_stamp":"2023-10-14T01:26:30.348012+0000","objects_scrubbed":1751,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":631,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:19:55.509715+0000","scrub_duration":630.66718906799997,"objects_trimmed":0,"snaptrim_duration":0.018770566999999998,"stat_sum":{"num_bytes":3676504619,"num_objects":1751,"num_object_clones":1105,"num_object_copies":3502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1751,"num_whiteouts":0,"num_read":324493,"num_read_kb":1277762,"num_write":322400,"num_write_kb":1289479,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":725,"num_omap_keys":11,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"145'348749","reported_seq":684653,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:31.731927+0000","last_change":"2023-10-14T01:26:29.185539+0000","last_active":"2023-10-14T01:35:31.731927+0000","last_peered":"2023-10-14T01:35:31.731927+0000","last_clean":"2023-10-14T01:35:31.731927+0000","last_became_active":"2023-10-13T23:59:43.807337+0000","last_became_peered":"2023-10-13T23:59:43.807337+0000","last_unstale":"2023-10-14T01:35:31.731927+0000","last_undegraded":"2023-10-14T01:35:31.731927+0000","last_fullsized":"2023-10-14T01:35:31.731927+0000","mapping_epoch":13,"log_start":"142'338713","ondisk_log_start":"142'338713","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"137'319563","last_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_deep_scrub":"137'319563","last_deep_scrub_stamp":"2023-10-14T01:26:29.185065+0000","last_clean_scrub_stamp":"2023-10-14T01:26:29.185065+0000","objects_scrubbed":1784,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":702,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:34:20.147022+0000","scrub_duration":701.95291060600005,"objects_trimmed":0,"snaptrim_duration":0.01879074,"stat_sum":{"num_bytes":3779473839,"num_objects":1784,"num_object_clones":1118,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":336115,"num_read_kb":1310619,"num_write":331953,"num_write_kb":1327494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2705,"num_omap_keys":73,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"145'334388","reported_seq":668177,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:30.443379+0000","last_change":"2023-10-14T01:14:44.846540+0000","last_active":"2023-10-14T01:35:30.443379+0000","last_peered":"2023-10-14T01:35:30.443379+0000","last_clean":"2023-10-14T01:35:30.443379+0000","last_became_active":"2023-10-13T23:59:43.805436+0000","last_became_peered":"2023-10-13T23:59:43.805436+0000","last_unstale":"2023-10-14T01:35:30.443379+0000","last_undegraded":"2023-10-14T01:35:30.443379+0000","last_fullsized":"2023-10-14T01:35:30.443379+0000","mapping_epoch":13,"log_start":"142'324312","ondisk_log_start":"142'324312","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"127'271916","last_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_deep_scrub":"127'271916","last_deep_scrub_stamp":"2023-10-14T01:14:44.846374+0000","last_clean_scrub_stamp":"2023-10-14T01:14:44.846374+0000","objects_scrubbed":1769,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":554,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T03:13:44.036461+0000","scrub_duration":553.85739751000006,"objects_trimmed":0,"snaptrim_duration":0.014390088000000001,"stat_sum":{"num_bytes":3720667607,"num_objects":1769,"num_object_clones":1112,"num_object_copies":3538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1769,"num_whiteouts":0,"num_read":333190,"num_read_kb":1291899,"num_write":326432,"num_write_kb":1305221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":4551,"num_omap_keys":73,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"145'353090","reported_seq":701739,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:34.847233+0000","last_change":"2023-10-14T01:15:54.515069+0000","last_active":"2023-10-14T01:35:34.847233+0000","last_peered":"2023-10-14T01:35:34.847233+0000","last_clean":"2023-10-14T01:35:34.847233+0000","last_became_active":"2023-10-13T23:59:43.803453+0000","last_became_peered":"2023-10-13T23:59:43.803453+0000","last_unstale":"2023-10-14T01:35:34.847233+0000","last_undegraded":"2023-10-14T01:35:34.847233+0000","last_fullsized":"2023-10-14T01:35:34.847233+0000","mapping_epoch":13,"log_start":"142'343012","ondisk_log_start":"142'343012","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'288973","last_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_deep_scrub":"128'288973","last_deep_scrub_stamp":"2023-10-14T01:15:54.514883+0000","last_clean_scrub_stamp":"2023-10-14T01:15:54.514883+0000","objects_scrubbed":1878,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":623,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T09:53:00.320138+0000","scrub_duration":622.42361405099996,"objects_trimmed":0,"snaptrim_duration":0.0018135899999999999,"stat_sum":{"num_bytes":3955601828,"num_objects":1878,"num_object_clones":1188,"num_object_copies":3756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1878,"num_whiteouts":0,"num_read":348813,"num_read_kb":1368992,"num_write":346185,"num_write_kb":1384566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"145'348457","reported_seq":691065,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:34.848828+0000","last_change":"2023-10-14T01:15:40.361006+0000","last_active":"2023-10-14T01:35:34.848828+0000","last_peered":"2023-10-14T01:35:34.848828+0000","last_clean":"2023-10-14T01:35:34.848828+0000","last_became_active":"2023-10-13T23:59:43.804750+0000","last_became_peered":"2023-10-13T23:59:43.804750+0000","last_unstale":"2023-10-14T01:35:34.848828+0000","last_undegraded":"2023-10-14T01:35:34.848828+0000","last_fullsized":"2023-10-14T01:35:34.848828+0000","mapping_epoch":13,"log_start":"142'338415","ondisk_log_start":"142'338415","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"128'285525","last_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_deep_scrub":"128'285525","last_deep_scrub_stamp":"2023-10-14T01:15:40.359653+0000","last_clean_scrub_stamp":"2023-10-14T01:15:40.359653+0000","objects_scrubbed":1860,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":607,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T11:54:35.976439+0000","scrub_duration":606.34314581299998,"objects_trimmed":0,"snaptrim_duration":0.0018549599999999999,"stat_sum":{"num_bytes":3898302817,"num_objects":1860,"num_object_clones":1180,"num_object_copies":3720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1860,"num_whiteouts":0,"num_read":342827,"num_read_kb":1345882,"num_write":341551,"num_write_kb":1366014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1233,"num_omap_keys":21,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"145'354732","reported_seq":694600,"reported_epoch":145,"state":"active+clean","last_fresh":"2023-10-14T01:35:31.731914+0000","last_change":"2023-10-14T01:28:25.512720+0000","last_active":"2023-10-14T01:35:31.731914+0000","last_peered":"2023-10-14T01:35:31.731914+0000","last_clean":"2023-10-14T01:35:31.731914+0000","last_became_active":"2023-10-13T23:59:43.807787+0000","last_became_peered":"2023-10-13T23:59:43.807787+0000","last_unstale":"2023-10-14T01:35:31.731914+0000","last_undegraded":"2023-10-14T01:35:31.731914+0000","last_fullsized":"2023-10-14T01:35:31.731914+0000","mapping_epoch":13,"log_start":"142'344717","ondisk_log_start":"142'344717","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'331319","last_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_deep_scrub":"139'331319","last_deep_scrub_stamp":"2023-10-14T01:28:25.512543+0000","last_clean_scrub_stamp":"2023-10-14T01:28:25.512543+0000","objects_scrubbed":1818,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":765,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T02:12:58.834979+0000","scrub_duration":764.70480949800003,"objects_trimmed":0,"snaptrim_duration":0.65201196500000003,"stat_sum":{"num_bytes":3805774260,"num_objects":1818,"num_object_clones":1150,"num_object_copies":3636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1818,"num_whiteouts":0,"num_read":339732,"num_read_kb":1324571,"num_write":333135,"num_write_kb":1332183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2682,"num_omap_keys":43,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"119'127","reported_seq":361,"reported_epoch":144,"state":"active+clean","last_fresh":"2023-10-14T01:34:08.989842+0000","last_change":"2023-10-14T01:05:31.470112+0000","last_active":"2023-10-14T01:34:08.989842+0000","last_peered":"2023-10-14T01:34:08.989842+0000","last_clean":"2023-10-14T01:34:08.989842+0000","last_became_active":"2023-10-13T23:59:38.932817+0000","last_became_peered":"2023-10-13T23:59:38.932817+0000","last_unstale":"2023-10-14T01:34:08.989842+0000","last_undegraded":"2023-10-14T01:34:08.989842+0000","last_fullsized":"2023-10-14T01:34:08.989842+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"119'127","last_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_deep_scrub":"119'127","last_deep_scrub_stamp":"2023-10-14T01:05:31.469910+0000","last_clean_scrub_stamp":"2023-10-14T01:05:31.469910+0000","objects_scrubbed":2,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-15T06:54:42.799309+0000","scrub_duration":0.033017139000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":29801053228,"num_objects":14179,"num_object_clones":8960,"num_object_copies":28358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14179,"num_whiteouts":0,"num_read":2643230,"num_read_kb":10338546,"num_write":2612479,"num_write_kb":10447833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":16715,"num_omap_keys":318,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21383561216,"data_stored":31253108824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38648,"internal_metadata":0},"log_size":80425,"ondisk_log_size":80425,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":169,"num_write_kb":2548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":12892,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":127,"ondisk_log_size":127,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359739514,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5781300,"kb_used_data":4082092,"kb_used_omap":534,"kb_used_meta":1698665,"kb_avail":88590540,"statfs":{"total":96636764160,"available":90716712960,"internally_reserved":0,"allocated":4180062208,"data_stored":6107005570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":547676,"internal_metadata":1739433124},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,3,5,8],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:35:17 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":1.0349999999999999,"15min":0.69799999999999995},"min":{"1min":0.188,"5min":0.188,"15min":0.182},"max":{"1min":7.2619999999999996,"5min":10.244,"15min":10.244},"last":0.26800000000000002},{"interface":"front","average":{"1min":2.2389999999999999,"5min":1.554,"15min":1.2569999999999999},"min":{"1min":0.223,"5min":0.20100000000000001,"15min":0.16400000000000001},"max":{"1min":12.574,"5min":16.097999999999999,"15min":16.097999999999999},"last":0.222}]},{"osd":1,"last update":"Sat Oct 14 01:35:17 2023","interfaces":[{"interface":"back","average":{"1min":2.0390000000000001,"5min":2.1230000000000002,"15min":1.827},"min":{"1min":0.23699999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":9.1129999999999995,"5min":20.43,"15min":34.158000000000001},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":1.0029999999999999,"15min":1.264},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":7.21,"5min":10.471,"15min":34.192999999999998},"last":0.30399999999999999}]}]},{"osd":1,"up_from":8,"seq":34359739501,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11458232,"kb_used_data":10441776,"kb_used_omap":835,"kb_used_meta":1015612,"kb_avail":82913608,"statfs":{"total":96636764160,"available":84903534592,"internally_reserved":0,"allocated":10692378624,"data_stored":15627000853,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":855304,"internal_metadata":1039987448},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,10,5,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Oct 14 01:34:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.52200000000000002,"15min":0.78500000000000003},"min":{"1min":0.223,"5min":0.19700000000000001,"15min":0.16600000000000001},"max":{"1min":1.6120000000000001,"5min":8.4469999999999992,"15min":16.481000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.76400000000000001,"15min":0.89000000000000001},"min":{"1min":0.26000000000000001,"5min":0.22,"15min":0.20499999999999999},"max":{"1min":3.331,"5min":8.407,"15min":16.52},"last":0.38300000000000001}]},{"osd":2,"last update":"Sat Oct 14 01:34:52 2023","interfaces":[{"interface":"back","average":{"1min":1.927,"5min":1.2390000000000001,"15min":1.016},"min":{"1min":0.26500000000000001,"5min":0.23100000000000001,"15min":0.21299999999999999},"max":{"1min":30.763999999999999,"5min":30.763999999999999,"15min":30.763999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":2.2250000000000001,"5min":1.0860000000000001,"15min":0.98199999999999998},"min":{"1min":0.23499999999999999,"5min":0.19500000000000001,"15min":0.193},"max":{"1min":26.440999999999999,"5min":26.440999999999999,"15min":26.440999999999999},"last":0.24099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359739515,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8348520,"kb_used_data":6361756,"kb_used_omap":790,"kb_used_meta":1985961,"kb_avail":86023320,"statfs":{"total":96636764160,"available":88087879680,"internally_reserved":0,"allocated":6514438144,"data_stored":9521350492,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":809695,"internal_metadata":2033624353},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Oct 14 01:34:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.64000000000000001,"15min":1.143},"min":{"1min":0.217,"5min":0.19400000000000001,"15min":0.191},"max":{"1min":4.391,"5min":5.3760000000000003,"15min":10.897},"last":0.5},{"interface":"front","average":{"1min":1.0840000000000001,"5min":1.7769999999999999,"15min":1.2909999999999999},"min":{"1min":0.182,"5min":0.182,"15min":0.182},"max":{"1min":3.5779999999999998,"5min":14.532999999999999,"15min":14.532999999999999},"last":0.315}]},{"osd":2,"last update":"Sat Oct 14 01:34:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.621,"15min":0.63400000000000001},"min":{"1min":0.254,"5min":0.20499999999999999,"15min":0.185},"max":{"1min":4.4400000000000004,"5min":5.4580000000000002,"15min":7.1319999999999997},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.90500000000000003,"15min":0.873},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":2.7770000000000001,"5min":6.1040000000000001,"15min":7.6289999999999996},"last":0.749}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":6446,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6513139712,"data_stored":9520240499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14948,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10691244032,"data_stored":15626042412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17003,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4179177472,"data_stored":6106825913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6697,"internal_metadata":0}]}} 2023-10-14T01:35:36.586 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-14T01:35:36.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.579+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:36.587 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.579+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:36.588 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.579+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:36.597 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:36.595+0000 7faeed24b640 -1 reset not still connected to 0x556f32d3bee0 2023-10-14T01:35:37.228 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:37.223+0000 7ff181d25640 -1 reset not still connected to 0x5628cd016a90 2023-10-14T01:35:37.253 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:37.251+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:37.253 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:37.251+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:37.531 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-14T01:35:37.532 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-14T01:35:37.532 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-14T01:35:37.532 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T01:35:37.588 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:37.587+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:37.594 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:37.591+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:40.740 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.735+0000 7faeed24b640 -1 reset not still connected to 0x556f31e38c30 2023-10-14T01:35:40.750 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31f33930 2023-10-14T01:35:40.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.743+0000 7faeed24b640 -1 reset not still connected to 0x556f31fa5a00 2023-10-14T01:35:40.751 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.743+0000 7faeed24b640 -1 reset not still connected to 0x556f3209c340 2023-10-14T01:35:40.752 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.743+0000 7faeed24b640 -1 reset not still connected to 0x556f32412410 2023-10-14T01:35:40.778 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.775+0000 7faeed24b640 -1 reset not still connected to 0x556f327f0c30 2023-10-14T01:35:40.785 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.779+0000 7faeed24b640 -1 reset not still connected to 0x556f33950750 2023-10-14T01:35:40.797 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.795+0000 7faeed24b640 -1 reset not still connected to 0x556f40f65860 2023-10-14T01:35:40.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.795+0000 7faeed24b640 -1 reset not still connected to 0x556f4727e270 2023-10-14T01:35:40.798 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.795+0000 7faeed24b640 -1 reset not still connected to 0x556f4727fad0 2023-10-14T01:35:40.808 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.803+0000 7faeed24b640 -1 reset not still connected to 0x556f4ea135f0 2023-10-14T01:35:40.809 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.803+0000 7faeed24b640 -1 reset not still connected to 0x556f54c2be10 2023-10-14T01:35:40.810 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f653804e0 2023-10-14T01:35:40.811 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.807+0000 7faeed24b640 -1 reset not still connected to 0x556f65381930 2023-10-14T01:35:40.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.823+0000 7faeed24b640 -1 reset not still connected to 0x556f6c228ea0 2023-10-14T01:35:40.828 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:40.823+0000 7faeed24b640 -1 reset not still connected to 0x556f6e42b790 2023-10-14T01:35:40.833 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-10-14T01:35:41.261 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:41.259+0000 7ff181d25640 -1 reset not still connected to 0x5628cdafadd0 2023-10-14T01:35:41.276 INFO:tasks.ceph.osd.2.smithi183.stderr:2023-10-14T01:35:41.271+0000 7ff181d25640 -1 reset not still connected to 0x5628cdb0b5f0 2023-10-14T01:35:41.348 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.347+0000 7faeed24b640 -1 reset not still connected to 0x556f4ea135f0 2023-10-14T01:35:41.368 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.363+0000 7faeed24b640 -1 reset not still connected to 0x556f54c2be10 2023-10-14T01:35:41.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.363+0000 7faeed24b640 -1 reset not still connected to 0x556f653804e0 2023-10-14T01:35:41.369 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.363+0000 7faeed24b640 -1 reset not still connected to 0x556f65381930 2023-10-14T01:35:41.382 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.379+0000 7faeed24b640 -1 reset not still connected to 0x556f6c228ea0 2023-10-14T01:35:41.384 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.379+0000 7faeed24b640 -1 reset not still connected to 0x556f6e42b790 2023-10-14T01:35:41.824 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.823+0000 7faeed24b640 -1 reset not still connected to 0x556f54c2be10 2023-10-14T01:35:41.834 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.831+0000 7faeed24b640 -1 reset not still connected to 0x556f653804e0 2023-10-14T01:35:41.835 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.831+0000 7faeed24b640 -1 reset not still connected to 0x556f65381930 2023-10-14T01:35:41.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.859+0000 7faeed24b640 -1 reset not still connected to 0x556f6c228ea0 2023-10-14T01:35:41.864 INFO:tasks.ceph.osd.1.smithi183.stderr:2023-10-14T01:35:41.859+0000 7faeed24b640 -1 reset not still connected to 0x556f6e42b790 2023-10-14T01:35:43.634 INFO:tasks.ceph.osd.0:Stopped 2023-10-14T01:35:43.635 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-14T01:35:43.635 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T01:35:46.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-10-14T01:35:46.138 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-10-14T01:35:49.737 INFO:tasks.ceph.osd.1:Stopped 2023-10-14T01:35:49.738 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-14T01:35:49.738 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T01:35:51.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-10-14T01:35:51.340 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-10-14T01:35:55.840 INFO:tasks.ceph.osd.2:Stopped 2023-10-14T01:35:55.841 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-14T01:35:55.841 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-14T01:35:55.842 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T01:35:56.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~0s 2023-10-14T01:35:56.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-10-14T01:36:01.544 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x is failed for ~5s 2023-10-14T01:36:01.944 INFO:tasks.ceph.mgr.x:Stopped 2023-10-14T01:36:01.945 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-14T01:36:01.945 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-14T01:36:01.946 INFO:teuthology.orchestra.run:waiting for 300 2023-10-14T01:36:06.546 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-10-14T01:36:06.546 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.x has been restored 2023-10-14T01:36:08.048 INFO:tasks.ceph.mon.a:Stopped 2023-10-14T01:36:08.048 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-14T01:36:08.049 DEBUG:teuthology.orchestra.run.smithi183:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | head -n 1 2023-10-14T01:36:08.145 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-14T01:36:08.145 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-14T01:36:08.838 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-14T01:36:08.839 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-14T01:36:09.647 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi183.front.sepia.ceph.com 2023-10-14T01:36:09.647 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-14T01:36:10.034 INFO:tasks.ceph:Archiving mon data... 2023-10-14T01:36:10.037 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259/data/mon.a.tgz 2023-10-14T01:36:10.038 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2023-10-14T01:36:10.043 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.5CuBCPmwsr 2023-10-14T01:36:10.044 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.5CuBCPmwsr 2023-10-14T01:36:10.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.5CuBCPmwsr 2023-10-14T01:36:10.302 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.5CuBCPmwsr is 573KB 2023-10-14T01:36:10.366 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.5CuBCPmwsr 2023-10-14T01:36:10.372 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-14T01:36:10.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-14T01:36:10.469 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-14T01:36:10.470 DEBUG:teuthology.orchestra.run.smithi183:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-14T01:36:10.510 INFO:teuthology.orchestra.run.smithi183.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-14T01:36:10.514 INFO:tasks.ceph:Archiving crash dumps... 2023-10-14T01:36:10.517 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/crash to /home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259/remote/smithi183/crash 2023-10-14T01:36:10.518 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-14T01:36:10.570 INFO:tasks.ceph:Compressing logs... 2023-10-14T01:36:10.571 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-14T01:36:10.640 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47382.log 2023-10-14T01:36:10.645 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53735.log 2023-10-14T01:36:10.646 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42859.log 2023-10-14T01:36:10.646 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48265.log 2023-10-14T01:36:10.646 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53735.log.gz 2023-10-14T01:36:10.647 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38725.log 2023-10-14T01:36:10.647 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42859.log: /var/log/ceph/ceph-client.admin.48265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55465.log 2023-10-14T01:36:10.647 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49359.log 2023-10-14T01:36:10.648 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38725.log.gz 2023-10-14T01:36:10.648 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48902.log 2023-10-14T01:36:10.648 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55465.log.gz 2023-10-14T01:36:10.649 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55570.log 2023-10-14T01:36:10.649 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49359.log: /var/log/ceph/ceph-client.admin.48902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51645.log 2023-10-14T01:36:10.649 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46925.log 2023-10-14T01:36:10.650 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55570.log.gz 2023-10-14T01:36:10.650 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40899.log 2023-10-14T01:36:10.650 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51645.log.gz 2023-10-14T01:36:10.651 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49451.log 2023-10-14T01:36:10.651 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38688.log 2023-10-14T01:36:10.651 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55326.log 2023-10-14T01:36:10.905 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42990.log 2023-10-14T01:36:10.906 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38688.log.gz 2023-10-14T01:36:10.906 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41363.log 2023-10-14T01:36:10.906 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55326.log.gz 2023-10-14T01:36:10.907 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46247.log 2023-10-14T01:36:10.907 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54823.log 2023-10-14T01:36:10.907 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51945.log 2023-10-14T01:36:10.908 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2023-10-14T01:36:10.908 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54823.log.gz 2023-10-14T01:36:10.908 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47840.log 2023-10-14T01:36:10.909 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51945.log.gz 2023-10-14T01:36:10.909 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46247.log.gz 0.0% 2023-10-14T01:36:10.909 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42859.log.gz 2023-10-14T01:36:10.910 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42990.log.gz 2023-10-14T01:36:10.910 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46925.log.gz 2023-10-14T01:36:10.910 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47382.log.gz 2023-10-14T01:36:10.911 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 2023-10-14T01:36:10.911 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41363.log.gz 2023-10-14T01:36:10.911 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48265.log.gz 2023-10-14T01:36:10.912 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49451.log.gz 2023-10-14T01:36:10.912 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48902.log.gz 2023-10-14T01:36:10.912 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49359.log.gz 2023-10-14T01:36:10.913 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37581.log 2023-10-14T01:36:10.913 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44525.log 2023-10-14T01:36:10.913 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47840.log.gz 2023-10-14T01:36:10.914 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45014.log 2023-10-14T01:36:10.914 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37581.log.gz 2023-10-14T01:36:10.914 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2023-10-14T01:36:10.915 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44525.log.gz 2023-10-14T01:36:10.915 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2023-10-14T01:36:10.915 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45014.log.gz 2023-10-14T01:36:10.916 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2023-10-14T01:36:10.916 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2023-10-14T01:36:10.916 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49329.log 2023-10-14T01:36:10.917 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2023-10-14T01:36:10.917 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46492.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2023-10-14T01:36:10.917 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39157.log 2023-10-14T01:36:10.917 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53552.log 2023-10-14T01:36:10.918 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49329.log.gz 2023-10-14T01:36:10.918 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43983.log 2023-10-14T01:36:10.918 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39157.log.gz 2023-10-14T01:36:10.919 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56237.log 2023-10-14T01:36:10.919 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53552.log.gz 2023-10-14T01:36:10.919 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45536.log 2023-10-14T01:36:10.919 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43983.log.gz 2023-10-14T01:36:10.920 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52361.log 2023-10-14T01:36:10.920 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56237.log.gz 2023-10-14T01:36:10.920 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52817.log 2023-10-14T01:36:10.920 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45536.log.gz 2023-10-14T01:36:10.921 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52895.log 2023-10-14T01:36:10.921 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52361.log.gz 2023-10-14T01:36:10.921 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39120.log 2023-10-14T01:36:10.921 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52817.log.gz 2023-10-14T01:36:10.922 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52895.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52895.log.gz -5 2023-10-14T01:36:10.922 INFO:teuthology.orchestra.run.smithi183.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2023-10-14T01:36:10.922 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41302.log 2023-10-14T01:36:10.923 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39120.log.gz 2023-10-14T01:36:10.923 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2023-10-14T01:36:10.923 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2023-10-14T01:36:10.923 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41143.log 2023-10-14T01:36:10.924 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41302.log.gz 2023-10-14T01:36:10.924 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39032.log 2023-10-14T01:36:10.924 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2023-10-14T01:36:10.924 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40899.log.gz 2023-10-14T01:36:10.925 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42257.log 2023-10-14T01:36:10.925 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41143.log.gz 2023-10-14T01:36:10.925 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43383.log 2023-10-14T01:36:10.925 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39032.log.gz 2023-10-14T01:36:10.926 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45978.log 2023-10-14T01:36:10.926 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42257.log.gz 2023-10-14T01:36:10.926 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41886.log 2023-10-14T01:36:10.927 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43383.log.gz 2023-10-14T01:36:10.927 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37583.log 2023-10-14T01:36:10.927 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45978.log.gz 2023-10-14T01:36:10.928 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46738.log 2023-10-14T01:36:10.928 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2023-10-14T01:36:10.928 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37583.log.gz 2023-10-14T01:36:10.929 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2023-10-14T01:36:10.929 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42704.log 2023-10-14T01:36:10.929 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46738.log.gz 2023-10-14T01:36:10.930 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53474.log 2023-10-14T01:36:10.930 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2023-10-14T01:36:10.930 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2023-10-14T01:36:10.931 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42704.log.gz 2023-10-14T01:36:10.931 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45627.log 2023-10-14T01:36:10.931 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53474.log.gz 2023-10-14T01:36:10.931 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47257.log 2023-10-14T01:36:10.932 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gz 2023-10-14T01:36:10.932 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50359.log 2023-10-14T01:36:10.933 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45627.log.gz 2023-10-14T01:36:10.933 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48749.log 2023-10-14T01:36:10.933 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47257.log.gz 2023-10-14T01:36:10.933 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45289.log 2023-10-14T01:36:10.934 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50359.log.gz 2023-10-14T01:36:10.934 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48749.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48749.log.gz 2023-10-14T01:36:10.935 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-14T01:36:10.935 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48082.log 2023-10-14T01:36:10.935 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45289.log.gz 2023-10-14T01:36:10.936 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51338.log 2023-10-14T01:36:10.936 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54667.log 2023-10-14T01:36:10.936 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48082.log.gz 2023-10-14T01:36:10.936 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46550.log 2023-10-14T01:36:10.937 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51338.log.gz 2023-10-14T01:36:10.937 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45597.log 2023-10-14T01:36:10.937 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54667.log.gz 2023-10-14T01:36:10.937 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39616.log 2023-10-14T01:36:10.938 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46550.log.gz 2023-10-14T01:36:10.938 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39584.log 2023-10-14T01:36:10.938 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45597.log.gz 2023-10-14T01:36:10.939 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47931.log 2023-10-14T01:36:10.939 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55084.log 2023-10-14T01:36:10.939 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47788.log 2023-10-14T01:36:10.940 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47931.log.gz 2023-10-14T01:36:10.940 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48933.log 2023-10-14T01:36:10.940 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55084.log.gz 2023-10-14T01:36:10.941 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38995.log 2023-10-14T01:36:10.941 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47788.log.gz 2023-10-14T01:36:10.941 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39584.log.gz 2023-10-14T01:36:10.942 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39616.log.gz 2023-10-14T01:36:10.942 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48933.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.48933.log.gz -- 2023-10-14T01:36:10.942 INFO:teuthology.orchestra.run.smithi183.stderr: /var/log/ceph/ceph-client.admin.42950.log 2023-10-14T01:36:10.943 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40246.log 2023-10-14T01:36:10.943 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38995.log.gz 2023-10-14T01:36:10.943 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47535.log 2023-10-14T01:36:10.944 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42950.log.gz 2023-10-14T01:36:10.944 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50510.log 2023-10-14T01:36:10.944 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44769.log 2023-10-14T01:36:10.945 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47535.log.gz 2023-10-14T01:36:10.945 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47694.log 2023-10-14T01:36:10.945 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50510.log.gz 2023-10-14T01:36:10.946 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2023-10-14T01:36:10.946 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44769.log.gz 2023-10-14T01:36:10.946 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40246.log.gz 2023-10-14T01:36:10.947 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39103.log 2023-10-14T01:36:10.947 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47694.log.gz 2023-10-14T01:36:10.947 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46366.log 2023-10-14T01:36:10.947 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2023-10-14T01:36:10.948 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44501.log 2023-10-14T01:36:10.948 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39103.log.gz 2023-10-14T01:36:10.948 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50209.log 2023-10-14T01:36:10.948 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46366.log.gz 2023-10-14T01:36:10.949 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44987.log 2023-10-14T01:36:10.949 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44501.log.gz 2023-10-14T01:36:10.949 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46642.log 2023-10-14T01:36:10.949 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50209.log.gz 2023-10-14T01:36:10.950 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44987.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.44987.log.gz --verbose 2023-10-14T01:36:10.950 INFO:teuthology.orchestra.run.smithi183.stderr: -- /var/log/ceph/ceph-client.admin.55058.log 2023-10-14T01:36:10.950 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41053.log 2023-10-14T01:36:10.951 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46642.log.gz 2023-10-14T01:36:10.951 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54050.log 2023-10-14T01:36:10.951 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55058.log.gz 2023-10-14T01:36:10.951 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41053.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41053.log.gz 2023-10-14T01:36:10.952 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43768.log 2023-10-14T01:36:10.952 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48416.log 2023-10-14T01:36:10.952 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54050.log.gz 2023-10-14T01:36:10.953 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54134.log 2023-10-14T01:36:10.953 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43768.log.gz 2023-10-14T01:36:10.953 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37661.log 2023-10-14T01:36:10.953 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48416.log.gz 2023-10-14T01:36:10.954 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43265.log 2023-10-14T01:36:10.954 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54134.log.gz 2023-10-14T01:36:10.954 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43289.log 2023-10-14T01:36:10.955 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37661.log.gz 2023-10-14T01:36:10.955 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39699.log 2023-10-14T01:36:10.955 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43265.log.gz 2023-10-14T01:36:10.955 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53868.log 2023-10-14T01:36:10.956 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43289.log.gz 2023-10-14T01:36:10.956 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38887.log 2023-10-14T01:36:10.956 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39699.log.gz 2023-10-14T01:36:10.957 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2023-10-14T01:36:10.957 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53868.log.gz 2023-10-14T01:36:10.957 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50179.log 2023-10-14T01:36:10.957 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38887.log.gz 2023-10-14T01:36:10.958 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41519.log 2023-10-14T01:36:10.958 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz 2023-10-14T01:36:10.958 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56101.log 2023-10-14T01:36:10.959 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50179.log.gz 2023-10-14T01:36:10.959 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37445.log 2023-10-14T01:36:10.959 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41519.log.gz 2023-10-14T01:36:10.960 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39410.log 2023-10-14T01:36:10.960 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56101.log.gz 2023-10-14T01:36:10.960 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51491.log 2023-10-14T01:36:10.961 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37445.log.gz 2023-10-14T01:36:10.961 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38526.log 2023-10-14T01:36:10.961 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39410.log.gz 2023-10-14T01:36:10.962 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34675.log 2023-10-14T01:36:10.962 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51491.log.gz 2023-10-14T01:36:10.962 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43133.log 2023-10-14T01:36:10.963 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38526.log.gz 2023-10-14T01:36:10.963 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.34675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49023.log 2023-10-14T01:36:10.963 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34675.log.gz 2023-10-14T01:36:10.964 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43133.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43133.log.gz -5 2023-10-14T01:36:10.964 INFO:teuthology.orchestra.run.smithi183.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2023-10-14T01:36:10.964 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54559.log 2023-10-14T01:36:10.965 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49023.log.gz 2023-10-14T01:36:10.965 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37523.log 2023-10-14T01:36:10.965 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2023-10-14T01:36:10.966 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48507.log 2023-10-14T01:36:10.966 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54559.log.gz 2023-10-14T01:36:10.966 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53341.log 2023-10-14T01:36:10.967 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37523.log.gz 2023-10-14T01:36:10.969 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38762.log 2023-10-14T01:36:10.970 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48507.log.gz 2023-10-14T01:36:10.970 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55649.log 2023-10-14T01:36:10.970 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53341.log.gz 2023-10-14T01:36:10.971 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38762.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38762.log.gz 2023-10-14T01:36:10.971 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42734.log 2023-10-14T01:36:10.971 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51836.log 2023-10-14T01:36:10.972 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55649.log.gz 2023-10-14T01:36:10.972 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48113.log 2023-10-14T01:36:10.972 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42734.log.gz 2023-10-14T01:36:10.973 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38816.log 2023-10-14T01:36:10.973 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51836.log.gz 2023-10-14T01:36:10.973 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44644.log 2023-10-14T01:36:10.974 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48113.log.gz 2023-10-14T01:36:10.974 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51158.log 2023-10-14T01:36:10.974 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38816.log.gz 2023-10-14T01:36:10.975 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2023-10-14T01:36:10.975 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49083.log 2023-10-14T01:36:10.975 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43352.log 2023-10-14T01:36:10.975 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2023-10-14T01:36:10.976 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48143.log 2023-10-14T01:36:10.976 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44829.log 2023-10-14T01:36:10.976 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37739.log 2023-10-14T01:36:10.976 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55596.log 2023-10-14T01:36:10.977 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48295.log 2023-10-14T01:36:10.977 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42039.log 2023-10-14T01:36:10.977 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55006.log 2023-10-14T01:36:10.978 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52712.log 2023-10-14T01:36:10.978 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53816.log 2023-10-14T01:36:10.978 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48993.log 2023-10-14T01:36:10.978 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52465.log 2023-10-14T01:36:10.979 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43633.log 2023-10-14T01:36:10.979 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42099.log 2023-10-14T01:36:10.979 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52439.log 2023-10-14T01:36:10.979 INFO:teuthology.orchestra.run.smithi183.stderr:gzip/var/log/ceph/ceph-client.admin.44644.log: -5 --verbose/var/log/ceph/ceph-client.admin.42099.log: -- /var/log/ceph/ceph-client.admin.51008.log 2023-10-14T01:36:10.980 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37731.log: /var/log/ceph/ceph-client.admin.51158.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42099.log.gz 2023-10-14T01:36:10.980 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49083.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2023-10-14T01:36:10.980 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51158.log.gz 2023-10-14T01:36:10.980 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49083.log.gz 2023-10-14T01:36:10.981 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2023-10-14T01:36:10.981 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-14T01:36:10.981 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52439.log.gz 2023-10-14T01:36:10.982 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48143.log.gz 2023-10-14T01:36:10.982 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2023-10-14T01:36:10.982 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51008.log: /var/log/ceph/ceph-client.admin.44829.log: /var/log/ceph/ceph-client.admin.37739.log: /var/log/ceph/ceph-client.admin.55596.log: /var/log/ceph/ceph-client.admin.48295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44829.log.gz/var/log/ceph/ceph-client.admin.42039.log: 2023-10-14T01:36:10.982 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37739.log.gz 2023-10-14T01:36:10.983 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55596.log.gz 2023-10-14T01:36:10.983 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48295.log.gz 2023-10-14T01:36:10.983 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42039.log.gz 2023-10-14T01:36:10.983 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44614.log 2023-10-14T01:36:10.984 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.55006.log: /var/log/ceph/ceph-client.admin.52712.log: /var/log/ceph/ceph-client.admin.53816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51008.log.gz 2023-10-14T01:36:10.984 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48993.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52712.log.gz 2023-10-14T01:36:10.984 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55006.log.gz 2023-10-14T01:36:10.985 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53816.log.gz 2023-10-14T01:36:10.985 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48993.log.gz 2023-10-14T01:36:10.985 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37809.log 2023-10-14T01:36:10.985 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52465.log: /var/log/ceph/ceph-client.admin.43633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44644.log.gz 2023-10-14T01:36:10.986 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52465.log.gz 2023-10-14T01:36:10.986 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43633.log.gz 2023-10-14T01:36:10.986 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56128.log 2023-10-14T01:36:10.987 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44614.log.gz 2023-10-14T01:36:10.987 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53288.log 2023-10-14T01:36:10.987 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37809.log.gz 2023-10-14T01:36:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2023-10-14T01:36:10.988 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56128.log.gz 2023-10-14T01:36:10.988 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49389.log 2023-10-14T01:36:10.988 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53288.log.gz 2023-10-14T01:36:10.989 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42367.log 2023-10-14T01:36:10.989 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2023-10-14T01:36:10.989 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54641.log 2023-10-14T01:36:10.990 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49389.log.gz 2023-10-14T01:36:10.990 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42367.log.gz 2023-10-14T01:36:10.990 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45950.log 2023-10-14T01:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53842.log 2023-10-14T01:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54641.log.gz 2023-10-14T01:36:10.991 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46119.log 2023-10-14T01:36:10.992 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45950.log.gz 2023-10-14T01:36:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53842.log.gz 2023-10-14T01:36:10.992 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41715.log 2023-10-14T01:36:10.993 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39265.log 2023-10-14T01:36:10.993 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46119.log.gz 2023-10-14T01:36:10.993 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49572.log 2023-10-14T01:36:10.994 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41715.log.gz 2023-10-14T01:36:10.994 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39265.log.gz 2023-10-14T01:36:10.994 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52130.log 2023-10-14T01:36:10.995 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49572.log.gz 2023-10-14T01:36:10.995 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43414.log 2023-10-14T01:36:10.996 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46089.log 2023-10-14T01:36:10.996 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52130.log.gz 2023-10-14T01:36:10.996 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42195.log 2023-10-14T01:36:10.997 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43414.log.gz 2023-10-14T01:36:10.997 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46089.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46089.log.gz 2023-10-14T01:36:10.997 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39066.log 2023-10-14T01:36:10.998 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41926.log 2023-10-14T01:36:10.998 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42195.log.gz 2023-10-14T01:36:10.998 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52571.log 2023-10-14T01:36:10.998 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39066.log.gz 2023-10-14T01:36:10.999 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41001.log 2023-10-14T01:36:10.999 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41926.log.gz 2023-10-14T01:36:10.999 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38870.log 2023-10-14T01:36:11.000 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52571.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52571.log.gz 2023-10-14T01:36:11.000 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47596.log 2023-10-14T01:36:11.000 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41001.log.gz 2023-10-14T01:36:11.001 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49482.log 2023-10-14T01:36:11.001 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38870.log.gz 2023-10-14T01:36:11.001 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45074.log 2023-10-14T01:36:11.002 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47596.log.gz 2023-10-14T01:36:11.002 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50672.log 2023-10-14T01:36:11.002 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49482.log.gz 2023-10-14T01:36:11.003 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52020.log 2023-10-14T01:36:11.003 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45074.log.gz 2023-10-14T01:36:11.003 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-14T01:36:11.004 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50672.log.gz 2023-10-14T01:36:11.004 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55357.log 2023-10-14T01:36:11.004 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52020.log.gz 2023-10-14T01:36:11.005 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47870.log 2023-10-14T01:36:11.005 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50450.log 2023-10-14T01:36:11.005 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55357.log.gz 2023-10-14T01:36:11.006 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51927.log 2023-10-14T01:36:11.006 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47870.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47870.log.gz 2023-10-14T01:36:11.006 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54850.log 2023-10-14T01:36:11.006 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50450.log.gz 2023-10-14T01:36:11.007 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51522.log 2023-10-14T01:36:11.007 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51927.log.gz 2023-10-14T01:36:11.007 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54850.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54850.log.gz -5 2023-10-14T01:36:11.007 INFO:teuthology.orchestra.run.smithi183.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43879.log 2023-10-14T01:36:11.008 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41982.log 2023-10-14T01:36:11.008 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51522.log.gz 2023-10-14T01:36:11.008 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41179.log 2023-10-14T01:36:11.008 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43879.log.gz 2023-10-14T01:36:11.009 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41982.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.41982.log.gz 2023-10-14T01:36:11.009 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49602.log 2023-10-14T01:36:11.009 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39918.log 2023-10-14T01:36:11.010 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41179.log.gz 2023-10-14T01:36:11.010 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48811.log 2023-10-14T01:36:11.010 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49602.log.gz 2023-10-14T01:36:11.010 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45180.log 2023-10-14T01:36:11.011 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55893.log 2023-10-14T01:36:11.011 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48811.log.gz 2023-10-14T01:36:11.011 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50058.log 2023-10-14T01:36:11.011 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45180.log.gz 2023-10-14T01:36:11.012 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43545.log 2023-10-14T01:36:11.012 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55893.log.gz 2023-10-14T01:36:11.012 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52791.log 2023-10-14T01:36:11.013 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50058.log.gz 2023-10-14T01:36:11.013 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53236.log 2023-10-14T01:36:11.013 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43545.log.gz 2023-10-14T01:36:11.013 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52206.log 2023-10-14T01:36:11.014 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52791.log.gz 2023-10-14T01:36:11.014 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45479.log 2023-10-14T01:36:11.014 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53236.log.gz 2023-10-14T01:36:11.014 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50796.log 2023-10-14T01:36:11.015 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52206.log.gz 2023-10-14T01:36:11.015 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40130.log 2023-10-14T01:36:11.015 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45479.log.gz 2023-10-14T01:36:11.016 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34436.log 2023-10-14T01:36:11.016 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50796.log.gz 2023-10-14T01:36:11.016 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42612.log 2023-10-14T01:36:11.016 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40130.log.gz 2023-10-14T01:36:11.017 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.34436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55758.log 2023-10-14T01:36:11.017 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34436.log.gz 2023-10-14T01:36:11.017 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53025.log 2023-10-14T01:36:11.018 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42612.log.gz 2023-10-14T01:36:11.018 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41578.log 2023-10-14T01:36:11.018 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55758.log.gz 2023-10-14T01:36:11.019 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48356.log 2023-10-14T01:36:11.019 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53025.log.gz 2023-10-14T01:36:11.019 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39012.log 2023-10-14T01:36:11.019 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41578.log.gz 2023-10-14T01:36:11.020 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38708.log 2023-10-14T01:36:11.020 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48356.log.gz 2023-10-14T01:36:11.021 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49148.log 2023-10-14T01:36:11.021 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39012.log.gz 2023-10-14T01:36:11.021 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38742.log 2023-10-14T01:36:11.022 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38708.log.gz 2023-10-14T01:36:11.022 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43018.log 2023-10-14T01:36:11.022 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49148.log.gz 2023-10-14T01:36:11.023 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41495.log 2023-10-14T01:36:11.023 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38742.log.gz 2023-10-14T01:36:11.023 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2023-10-14T01:36:11.024 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43018.log.gz 2023-10-14T01:36:11.024 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37420.log 2023-10-14T01:36:11.024 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41495.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41495.log.gz 2023-10-14T01:36:11.025 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47505.log 2023-10-14T01:36:11.025 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2023-10-14T01:36:11.025 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37420.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37420.log.gz 2023-10-14T01:36:11.026 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50149.log 2023-10-14T01:36:11.026 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42890.log 2023-10-14T01:36:11.026 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47505.log.gz 2023-10-14T01:36:11.027 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50609.log 2023-10-14T01:36:11.027 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50149.log.gz 2023-10-14T01:36:11.027 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46855.log 2023-10-14T01:36:11.028 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42890.log.gz 2023-10-14T01:36:11.028 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44398.log 2023-10-14T01:36:11.028 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50609.log.gz 2023-10-14T01:36:11.029 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54242.log 2023-10-14T01:36:11.029 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46855.log.gz 2023-10-14T01:36:11.029 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38418.log 2023-10-14T01:36:11.030 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44398.log.gz 2023-10-14T01:36:11.030 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44035.log 2023-10-14T01:36:11.030 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54242.log.gz 2023-10-14T01:36:11.031 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54693.log 2023-10-14T01:36:11.031 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38418.log.gz 2023-10-14T01:36:11.031 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49845.log 2023-10-14T01:36:11.032 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44035.log.gz 2023-10-14T01:36:11.032 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42288.log 2023-10-14T01:36:11.032 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54693.log.gz 2023-10-14T01:36:11.033 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40686.log 2023-10-14T01:36:11.033 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49845.log.gz 2023-10-14T01:36:11.033 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53314.log 2023-10-14T01:36:11.034 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42288.log.gz 2023-10-14T01:36:11.034 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55785.log 2023-10-14T01:36:11.034 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40686.log.gz 2023-10-14T01:36:11.035 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54771.log 2023-10-14T01:36:11.035 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53314.log.gz 2023-10-14T01:36:11.035 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39952.log 2023-10-14T01:36:11.036 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55785.log.gz 2023-10-14T01:36:11.036 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48658.log 2023-10-14T01:36:11.036 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54771.log.gz 2023-10-14T01:36:11.036 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54269.log 2023-10-14T01:36:11.037 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54375.log 2023-10-14T01:36:11.037 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48658.log.gz 2023-10-14T01:36:11.037 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53158.log 2023-10-14T01:36:11.037 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54269.log.gz 2023-10-14T01:36:11.038 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37471.log 2023-10-14T01:36:11.038 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54375.log.gz 2023-10-14T01:36:11.038 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46795.log 2023-10-14T01:36:11.039 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53158.log.gz 2023-10-14T01:36:11.039 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47230.log 2023-10-14T01:36:11.039 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37471.log.gz 2023-10-14T01:36:11.039 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51248.log 2023-10-14T01:36:11.040 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46795.log.gz 2023-10-14T01:36:11.040 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51098.log 2023-10-14T01:36:11.040 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47230.log.gz 2023-10-14T01:36:11.041 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40809.log 2023-10-14T01:36:11.041 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51248.log.gz 2023-10-14T01:36:11.041 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42561.log 2023-10-14T01:36:11.041 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51098.log.gz 2023-10-14T01:36:11.042 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55413.log 2023-10-14T01:36:11.042 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40809.log.gz 2023-10-14T01:36:11.042 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50269.log 2023-10-14T01:36:11.043 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42561.log.gz 2023-10-14T01:36:11.043 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55413.log.gz 2023-10-14T01:36:11.043 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51188.log 2023-10-14T01:36:11.043 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39918.log.gz 2023-10-14T01:36:11.044 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50269.log.gz 2023-10-14T01:36:11.044 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39952.log.gz 2023-10-14T01:36:11.044 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2023-10-14T01:36:11.045 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52413.log 2023-10-14T01:36:11.045 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51188.log.gz 2023-10-14T01:36:11.045 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51399.log 2023-10-14T01:36:11.046 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2023-10-14T01:36:11.046 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38509.log 2023-10-14T01:36:11.046 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52413.log.gz 2023-10-14T01:36:11.047 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50577.log 0.0% 2023-10-14T01:36:11.047 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51399.log.gz 2023-10-14T01:36:11.047 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45232.log 2023-10-14T01:36:11.048 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38509.log.gz 2023-10-14T01:36:11.048 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41826.log 2023-10-14T01:36:11.048 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50577.log.gz 2023-10-14T01:36:11.049 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44933.log 2023-10-14T01:36:11.049 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45232.log.gz 2023-10-14T01:36:11.049 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39174.log 2023-10-14T01:36:11.050 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41826.log.gz 2023-10-14T01:36:11.050 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41467.log 2023-10-14T01:36:11.050 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44933.log.gz 2023-10-14T01:36:11.051 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45044.log 2023-10-14T01:36:11.051 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39174.log.gz 2023-10-14T01:36:11.051 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50028.log 2023-10-14T01:36:11.052 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41467.log.gz 2023-10-14T01:36:11.052 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40078.log 2023-10-14T01:36:11.052 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45044.log.gz 2023-10-14T01:36:11.053 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2023-10-14T01:36:11.053 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50028.log.gz 2023-10-14T01:36:11.053 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37861.log 2023-10-14T01:36:11.054 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40078.log.gz 2023-10-14T01:36:11.054 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38978.log 2023-10-14T01:36:11.054 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2023-10-14T01:36:11.055 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41025.log 2023-10-14T01:36:11.055 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37861.log.gz 2023-10-14T01:36:11.055 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48022.log 2023-10-14T01:36:11.056 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38978.log.gz 2023-10-14T01:36:11.056 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49632.log 2023-10-14T01:36:11.056 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41025.log.gz 2023-10-14T01:36:11.057 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51461.log 2023-10-14T01:36:11.057 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48022.log.gz 2023-10-14T01:36:11.057 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40869.log 2023-10-14T01:36:11.058 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49632.log.gz 2023-10-14T01:36:11.058 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46520.log 2023-10-14T01:36:11.058 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51461.log.gz 2023-10-14T01:36:11.059 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52947.log 2023-10-14T01:36:11.059 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40869.log.gz 2023-10-14T01:36:11.059 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2023-10-14T01:36:11.060 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46520.log.gz 2023-10-14T01:36:11.060 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44685.log 2023-10-14T01:36:11.060 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52947.log.gz 2023-10-14T01:36:11.061 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54745.log 2023-10-14T01:36:11.061 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2023-10-14T01:36:11.061 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50978.log 2023-10-14T01:36:11.062 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44685.log.gz 2023-10-14T01:36:11.062 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44093.log 2023-10-14T01:36:11.062 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54745.log.gz 2023-10-14T01:36:11.062 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38634.log 2023-10-14T01:36:11.063 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50978.log.gz 2023-10-14T01:36:11.063 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53761.log 2023-10-14T01:36:11.063 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44093.log.gz 2023-10-14T01:36:11.063 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40973.log 2023-10-14T01:36:11.064 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38634.log.gz 2023-10-14T01:36:11.064 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45208.log 2023-10-14T01:36:11.064 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53761.log.gz 2023-10-14T01:36:11.065 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40730.log 2023-10-14T01:36:11.065 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40973.log.gz 2023-10-14T01:36:11.065 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44255.log 2023-10-14T01:36:11.065 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45208.log.gz 2023-10-14T01:36:11.066 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45842.log 2023-10-14T01:36:11.066 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40730.log.gz 2023-10-14T01:36:11.066 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2023-10-14T01:36:11.067 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44255.log.gz 2023-10-14T01:36:11.067 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48567.log 2023-10-14T01:36:11.067 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45842.log.gz 2023-10-14T01:36:11.067 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53262.log 2023-10-14T01:36:11.068 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2023-10-14T01:36:11.068 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44799.log 2023-10-14T01:36:11.068 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48567.log.gz 2023-10-14T01:36:11.069 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40446.log 2023-10-14T01:36:11.069 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53262.log.gz 2023-10-14T01:36:11.069 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45702.log 2023-10-14T01:36:11.069 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44799.log.gz 2023-10-14T01:36:11.070 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54980.log 2023-10-14T01:36:11.070 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40446.log.gz 2023-10-14T01:36:11.070 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56049.log 2023-10-14T01:36:11.071 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45702.log.gz 2023-10-14T01:36:11.071 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54980.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54980.log.gz 2023-10-14T01:36:11.071 INFO:teuthology.orchestra.run.smithi183.stderr: /var/log/ceph/ceph-client.admin.40422.log 2023-10-14T01:36:11.071 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43939.log 2023-10-14T01:36:11.072 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56049.log.gz 2023-10-14T01:36:11.072 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49662.log 2023-10-14T01:36:11.072 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40422.log.gz 2023-10-14T01:36:11.072 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53500.log 2023-10-14T01:36:11.073 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43939.log.gz 2023-10-14T01:36:11.073 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47317.log 2023-10-14T01:36:11.073 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49662.log.gz 2023-10-14T01:36:11.073 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51615.log 2023-10-14T01:36:11.074 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53500.log.gz 2023-10-14T01:36:11.074 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55491.log 2023-10-14T01:36:11.074 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47317.log.gz 2023-10-14T01:36:11.075 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40172.log 2023-10-14T01:36:11.075 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51615.log.gz 2023-10-14T01:36:11.075 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55142.log 2023-10-14T01:36:11.076 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55491.log.gz 2023-10-14T01:36:11.076 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54797.log 2023-10-14T01:36:11.076 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40172.log.gz 2023-10-14T01:36:11.077 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38563.log 2023-10-14T01:36:11.077 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55142.log.gz 2023-10-14T01:36:11.077 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50239.log 2023-10-14T01:36:11.077 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54797.log.gz 2023-10-14T01:36:11.078 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44279.log 2023-10-14T01:36:11.078 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38563.log.gz 2023-10-14T01:36:11.079 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53106.log 2023-10-14T01:36:11.079 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50239.log.gz 2023-10-14T01:36:11.079 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40626.log 2023-10-14T01:36:11.079 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44279.log.gz 2023-10-14T01:36:11.080 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41954.log 2023-10-14T01:36:11.080 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53106.log.gz 2023-10-14T01:36:11.081 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47901.log 2023-10-14T01:36:11.081 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40626.log.gz 2023-10-14T01:36:11.081 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40782.log 2023-10-14T01:36:11.082 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41954.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41954.log.gz 2023-10-14T01:36:11.082 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44153.log 2023-10-14T01:36:11.082 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47901.log.gz 2023-10-14T01:36:11.083 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40782.log.gz 2023-10-14T01:36:11.083 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47454.log 2023-10-14T01:36:11.083 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44153.log.gz 2023-10-14T01:36:11.084 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45754.log 2023-10-14T01:36:11.084 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-14T01:36:11.084 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2023-10-14T01:36:11.085 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45754.log.gz 2023-10-14T01:36:11.085 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54928.log 2023-10-14T01:36:11.085 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39319.log 2023-10-14T01:36:11.086 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39464.log 2023-10-14T01:36:11.086 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54928.log.gz 2023-10-14T01:36:11.086 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37497.log 2023-10-14T01:36:11.087 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39319.log.gz 2023-10-14T01:36:11.087 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39464.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39464.log.gz 2023-10-14T01:36:11.087 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50826.log 2023-10-14T01:36:11.088 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43819.log 2023-10-14T01:36:11.088 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37497.log.gz 2023-10-14T01:36:11.088 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55248.log 2023-10-14T01:36:11.089 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50826.log.gz 2023-10-14T01:36:11.089 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52110.log 2023-10-14T01:36:11.089 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43819.log.gz 2023-10-14T01:36:11.090 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42533.log 2023-10-14T01:36:11.090 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55248.log.gz 2023-10-14T01:36:11.090 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48386.log 2023-10-14T01:36:11.091 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52110.log.gz 2023-10-14T01:36:11.091 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54587.log 2023-10-14T01:36:11.091 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42533.log.gz 2023-10-14T01:36:11.092 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45567.log 2023-10-14T01:36:11.092 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48386.log.gz 2023-10-14T01:36:11.092 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52685.log 2023-10-14T01:36:11.092 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54587.log.gz 2023-10-14T01:36:11.093 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2023-10-14T01:36:11.093 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45567.log.gz 2023-10-14T01:36:11.093 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51909.log 2023-10-14T01:36:11.093 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52685.log.gz 2023-10-14T01:36:11.094 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41795.log 2023-10-14T01:36:11.094 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2023-10-14T01:36:11.094 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39518.log 2023-10-14T01:36:11.094 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51909.log.gz 2023-10-14T01:36:11.095 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41795.log.gz 2023-10-14T01:36:11.095 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51818.log 2023-10-14T01:36:11.095 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54213.log 2023-10-14T01:36:11.096 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39518.log.gz 2023-10-14T01:36:11.096 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51818.log.gz 2023-10-14T01:36:11.096 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52148.log 2023-10-14T01:36:11.096 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52283.log 2023-10-14T01:36:11.097 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54213.log.gz 2023-10-14T01:36:11.097 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46686.log 2023-10-14T01:36:11.097 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52148.log.gz 2023-10-14T01:36:11.097 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53393.log 2023-10-14T01:36:11.098 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52283.log.gz 2023-10-14T01:36:11.098 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38435.log 2023-10-14T01:36:11.098 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46686.log.gz 2023-10-14T01:36:11.098 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47666.log 2023-10-14T01:36:11.099 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53393.log.gz 2023-10-14T01:36:11.099 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54323.log 2023-10-14T01:36:11.099 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2023-10-14T01:36:11.100 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43572.log 2023-10-14T01:36:11.100 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47666.log.gz 2023-10-14T01:36:11.100 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2023-10-14T01:36:11.100 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54323.log.gz 2023-10-14T01:36:11.101 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50389.log 2023-10-14T01:36:11.101 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43572.log.gz 2023-10-14T01:36:11.101 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39211.log 2023-10-14T01:36:11.101 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2023-10-14T01:36:11.102 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53972.log 2023-10-14T01:36:11.102 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50389.log.gz 2023-10-14T01:36:11.102 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43321.log 2023-10-14T01:36:11.103 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39211.log.gz 2023-10-14T01:36:11.103 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39784.log 2023-10-14T01:36:11.103 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53972.log.gz 2023-10-14T01:36:11.104 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56287.log 2023-10-14T01:36:11.104 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2023-10-14T01:36:11.104 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40004.log 2023-10-14T01:36:11.105 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39784.log.gz 2023-10-14T01:36:11.105 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42069.log 2023-10-14T01:36:11.105 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56287.log.gz 2023-10-14T01:36:11.106 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39427.log 2023-10-14T01:36:11.106 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40004.log.gz 2023-10-14T01:36:11.106 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52387.log 2023-10-14T01:36:11.107 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2023-10-14T01:36:11.107 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45383.log 2023-10-14T01:36:11.107 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39427.log.gz 2023-10-14T01:36:11.108 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52038.log 2023-10-14T01:36:11.108 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52387.log.gz 2023-10-14T01:36:11.108 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51068.log 2023-10-14T01:36:11.109 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45383.log.gz 2023-10-14T01:36:11.109 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51782.log 2023-10-14T01:36:11.109 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52038.log.gz 2023-10-14T01:36:11.110 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51068.log.gz 2023-10-14T01:36:11.110 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2023-10-14T01:36:11.110 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44063.log 2023-10-14T01:36:11.111 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51782.log.gz 2023-10-14T01:36:11.111 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52544.log 2023-10-14T01:36:11.111 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2023-10-14T01:36:11.112 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54108.log 2023-10-14T01:36:11.112 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44063.log.gz 2023-10-14T01:36:11.112 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52544.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.52544.log.gz --verbose 2023-10-14T01:36:11.113 INFO:teuthology.orchestra.run.smithi183.stderr: -- /var/log/ceph/ceph-client.admin.51873.log 2023-10-14T01:36:11.113 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41669.log 2023-10-14T01:36:11.113 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54108.log.gz 2023-10-14T01:36:11.114 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2023-10-14T01:36:11.114 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51873.log.gz 2023-10-14T01:36:11.114 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51278.log 2023-10-14T01:36:11.115 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41669.log.gz 2023-10-14T01:36:11.115 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41768.log 2023-10-14T01:36:11.115 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2023-10-14T01:36:11.116 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55544.log 2023-10-14T01:36:11.116 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51278.log.gz 2023-10-14T01:36:11.116 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43237.log 2023-10-14T01:36:11.117 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41768.log.gz 2023-10-14T01:36:11.117 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53077.log 2023-10-14T01:36:11.117 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55544.log.gz 2023-10-14T01:36:11.118 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49723.log 2023-10-14T01:36:11.118 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43237.log.gz 2023-10-14T01:36:11.118 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37738.log 2023-10-14T01:36:11.119 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53077.log.gz 2023-10-14T01:36:11.119 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41113.log 2023-10-14T01:36:11.119 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49723.log.gz 2023-10-14T01:36:11.120 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41548.log 2023-10-14T01:36:11.120 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37738.log.gz 2023-10-14T01:36:11.120 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49239.log 2023-10-14T01:36:11.120 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41113.log.gz 2023-10-14T01:36:11.121 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53578.log 2023-10-14T01:36:11.121 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41548.log.gz 2023-10-14T01:36:11.121 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2023-10-14T01:36:11.121 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49239.log.gz 2023-10-14T01:36:11.122 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51308.log 2023-10-14T01:36:11.122 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53578.log.gz 2023-10-14T01:36:11.122 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51763.log 2023-10-14T01:36:11.122 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2023-10-14T01:36:11.123 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41608.log 2023-10-14T01:36:11.123 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51308.log.gz 2023-10-14T01:36:11.123 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51763.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-14T01:36:11.124 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51763.log.gz 2023-10-14T01:36:11.124 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54024.log 2023-10-14T01:36:11.124 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41608.log.gz 2023-10-14T01:36:11.124 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43193.log 2023-10-14T01:36:11.125 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47626.log 2023-10-14T01:36:11.125 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54024.log.gz 2023-10-14T01:36:11.125 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44473.log 2023-10-14T01:36:11.125 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43193.log.gz 2023-10-14T01:36:11.126 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39049.log 2023-10-14T01:36:11.126 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47626.log.gz 2023-10-14T01:36:11.126 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52309.log 2023-10-14T01:36:11.127 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44473.log.gz 2023-10-14T01:36:11.127 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40596.log 2023-10-14T01:36:11.127 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2023-10-14T01:36:11.127 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55385.log 2023-10-14T01:36:11.128 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52309.log.gz 2023-10-14T01:36:11.128 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38958.log 2023-10-14T01:36:11.128 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40596.log.gz 2023-10-14T01:36:11.129 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47039.log 2023-10-14T01:36:11.129 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55385.log.gz 2023-10-14T01:36:11.129 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51585.log 2023-10-14T01:36:11.129 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38958.log.gz 2023-10-14T01:36:11.130 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5/var/log/ceph/ceph-client.admin.47039.log: --verbose -- /var/log/ceph/ceph-client.admin.40656.log 2023-10-14T01:36:11.130 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47039.log.gz 2023-10-14T01:36:11.130 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53526.log 2023-10-14T01:36:11.131 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51585.log.gz 2023-10-14T01:36:11.131 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40656.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-14T01:36:11.131 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40656.log.gz 2023-10-14T01:36:11.132 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51430.log 2023-10-14T01:36:11.132 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53526.log.gz 2023-10-14T01:36:11.132 INFO:teuthology.orchestra.run.smithi183.stderr:gzip/var/log/ceph/ceph.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52999.log 2023-10-14T01:36:11.132 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49998.log 2023-10-14T01:36:11.133 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51430.log.gz 2023-10-14T01:36:11.133 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54507.log 2023-10-14T01:36:11.133 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52999.log.gz 2023-10-14T01:36:11.134 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51705.log 2023-10-14T01:36:11.134 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49998.log.gz 2023-10-14T01:36:11.134 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50299.log 2023-10-14T01:36:11.135 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54507.log.gz 2023-10-14T01:36:11.135 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51705.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51705.log.gz 2023-10-14T01:36:11.135 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54453.log 2023-10-14T01:36:11.136 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43492.log 2023-10-14T01:36:11.136 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50299.log.gz 2023-10-14T01:36:11.136 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2023-10-14T01:36:11.137 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54453.log.gz 2023-10-14T01:36:11.137 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55622.log 2023-10-14T01:36:11.137 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43492.log.gz 2023-10-14T01:36:11.138 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44337.log 2023-10-14T01:36:11.138 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz 2023-10-14T01:36:11.138 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2023-10-14T01:36:11.139 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55622.log.gz 2023-10-14T01:36:11.139 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44337.log.gz 2023-10-14T01:36:11.139 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40296.log 2023-10-14T01:36:11.140 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38452.log 2023-10-14T01:36:11.140 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2023-10-14T01:36:11.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47008.log 2023-10-14T01:36:11.141 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40296.log.gz 2023-10-14T01:36:11.141 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53708.log 2023-10-14T01:36:11.142 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38452.log.gz 2023-10-14T01:36:11.142 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47008.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47008.log.gz 2023-10-14T01:36:11.142 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46611.log 2023-10-14T01:36:11.143 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2023-10-14T01:36:11.143 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53708.log.gz 2023-10-14T01:36:11.143 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46611.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.46611.log.gz 2023-10-14T01:36:11.144 INFO:teuthology.orchestra.run.smithi183.stderr: /var/log/ceph/ceph-client.admin.48537.log 2023-10-14T01:36:11.144 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40487.log 2023-10-14T01:36:11.144 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2023-10-14T01:36:11.145 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48537.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48537.log.gz 2023-10-14T01:36:11.145 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49753.log 2023-10-14T01:36:11.145 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54615.log 2023-10-14T01:36:11.146 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40487.log.gz 2023-10-14T01:36:11.146 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39648.log 2023-10-14T01:36:11.146 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49753.log.gz 2023-10-14T01:36:11.147 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53998.log 2023-10-14T01:36:11.147 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54615.log.gz 2023-10-14T01:36:11.147 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52765.log 2023-10-14T01:36:11.148 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39648.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39648.log.gz 2023-10-14T01:36:11.148 INFO:teuthology.orchestra.run.smithi183.stderr: 89.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-14T01:36:11.148 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44227.log 2023-10-14T01:36:11.148 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53998.log.gz 2023-10-14T01:36:11.149 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52765.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52765.log.gz 2023-10-14T01:36:11.149 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39356.log 2023-10-14T01:36:11.149 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55997.log 2023-10-14T01:36:11.150 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44227.log.gz 2023-10-14T01:36:11.150 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48477.log 2023-10-14T01:36:11.150 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39356.log.gz 2023-10-14T01:36:11.150 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52335.log 2023-10-14T01:36:11.151 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55997.log.gz 2023-10-14T01:36:11.151 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47071.log 2023-10-14T01:36:11.151 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48477.log.gz 2023-10-14T01:36:11.152 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46002.log 2023-10-14T01:36:11.152 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52335.log.gz 2023-10-14T01:36:11.152 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42585.log 2023-10-14T01:36:11.152 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47071.log.gz 2023-10-14T01:36:11.153 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46002.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46002.log.gz 2023-10-14T01:36:11.153 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46222.log 2023-10-14T01:36:11.153 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50420.log 2023-10-14T01:36:11.153 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42585.log.gz 2023-10-14T01:36:11.154 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55945.log 2023-10-14T01:36:11.154 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46222.log.gz 2023-10-14T01:36:11.154 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54080.log 2023-10-14T01:36:11.154 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50420.log.gz 2023-10-14T01:36:11.155 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2023-10-14T01:36:11.155 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55945.log.gz 2023-10-14T01:36:11.155 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39373.log 2023-10-14T01:36:11.156 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54080.log.gz 2023-10-14T01:36:11.156 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45455.log 2023-10-14T01:36:11.156 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2023-10-14T01:36:11.156 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34578.log 2023-10-14T01:36:11.157 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39373.log.gz 2023-10-14T01:36:11.157 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50948.log 2023-10-14T01:36:11.157 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45455.log.gz 2023-10-14T01:36:11.157 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.34578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54186.log 2023-10-14T01:36:11.158 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34578.log.gz 2023-10-14T01:36:11.158 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55919.log 2023-10-14T01:36:11.158 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50948.log.gz 2023-10-14T01:36:11.159 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45106.log 2023-10-14T01:36:11.159 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54186.log.gz 2023-10-14T01:36:11.159 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2023-10-14T01:36:11.160 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55919.log.gz 2023-10-14T01:36:11.160 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49114.log 2023-10-14T01:36:11.160 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45106.log.gz 2023-10-14T01:36:11.160 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55196.log 2023-10-14T01:36:11.161 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2023-10-14T01:36:11.161 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53184.log 2023-10-14T01:36:11.161 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49114.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49114.log.gz 2023-10-14T01:36:11.162 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39248.log 2023-10-14T01:36:11.162 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55196.log.gz 2023-10-14T01:36:11.162 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53184.log.gz 2023-10-14T01:36:11.163 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39282.log 2023-10-14T01:36:11.163 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39248.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39248.log.gz 2023-10-14T01:36:11.163 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50480.log 2023-10-14T01:36:11.164 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44962.log 2023-10-14T01:36:11.164 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39282.log.gz 2023-10-14T01:36:11.164 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34530.log 2023-10-14T01:36:11.165 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50480.log.gz 2023-10-14T01:36:11.165 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45730.log 2023-10-14T01:36:11.165 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44962.log.gz 2023-10-14T01:36:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.34530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44306.log 2023-10-14T01:36:11.166 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34530.log.gz 2023-10-14T01:36:11.166 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51891.log 2023-10-14T01:36:11.167 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45730.log.gz 2023-10-14T01:36:11.167 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42340.log 2023-10-14T01:36:11.167 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44306.log.gz 2023-10-14T01:36:11.168 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44183.log 2023-10-14T01:36:11.168 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51891.log.gz 2023-10-14T01:36:11.169 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55116.log 2023-10-14T01:36:11.169 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42340.log.gz 2023-10-14T01:36:11.169 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38924.log 2023-10-14T01:36:11.170 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44183.log.gz 2023-10-14T01:36:11.170 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46304.log 2023-10-14T01:36:11.170 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55116.log.gz 2023-10-14T01:36:11.170 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48597.log 2023-10-14T01:36:11.171 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38924.log.gz 2023-10-14T01:36:11.171 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46274.log 2023-10-14T01:36:11.172 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46304.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46304.log.gz 2023-10-14T01:36:11.172 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43046.log 2023-10-14T01:36:11.172 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48597.log.gz 2023-10-14T01:36:11.172 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55676.log 2023-10-14T01:36:11.173 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46274.log.gz 2023-10-14T01:36:11.173 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46059.log 2023-10-14T01:36:11.174 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43046.log.gz 2023-10-14T01:36:11.174 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2023-10-14T01:36:11.174 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55676.log.gz 2023-10-14T01:36:11.175 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46059.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46059.log.gzgzip 2023-10-14T01:36:11.175 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39498.log 2023-10-14T01:36:11.175 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47992.log 2023-10-14T01:36:11.176 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2023-10-14T01:36:11.176 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42316.log 2023-10-14T01:36:11.176 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39498.log.gz 2023-10-14T01:36:11.176 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47992.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.47992.log.gz 2023-10-14T01:36:11.177 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37657.log 2023-10-14T01:36:11.177 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49179.log 2023-10-14T01:36:11.177 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42316.log.gz 2023-10-14T01:36:11.177 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51675.log 2023-10-14T01:36:11.178 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37657.log.gz 2023-10-14T01:36:11.178 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39086.log 2023-10-14T01:36:11.178 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49179.log.gz 2023-10-14T01:36:11.178 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39868.log 2023-10-14T01:36:11.179 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51675.log.gz 2023-10-14T01:36:11.179 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45812.log 2023-10-14T01:36:11.179 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39086.log.gz 2023-10-14T01:36:11.180 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49053.log 2023-10-14T01:36:11.180 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38850.log 2023-10-14T01:36:11.180 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45812.log.gz 2023-10-14T01:36:11.180 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52597.log 2023-10-14T01:36:11.181 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49053.log.gz 2023-10-14T01:36:11.181 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48325.log 2023-10-14T01:36:11.181 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38850.log.gz 2023-10-14T01:36:11.181 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50641.log 2023-10-14T01:36:11.182 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52597.log.gz 2023-10-14T01:36:11.182 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43444.log 2023-10-14T01:36:11.182 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48325.log.gz 2023-10-14T01:36:11.182 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39868.log.gz 2023-10-14T01:36:11.183 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2023-10-14T01:36:11.183 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50641.log.gz 2023-10-14T01:36:11.183 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51800.log 2023-10-14T01:36:11.183 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43444.log.gz 2023-10-14T01:36:11.184 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49785.log 2023-10-14T01:36:11.184 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2023-10-14T01:36:11.184 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51800.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.51800.log.gz -- 2023-10-14T01:36:11.184 INFO:teuthology.orchestra.run.smithi183.stderr: /var/log/ceph/ceph-client.admin.48718.log 2023-10-14T01:36:11.185 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38546.log 2023-10-14T01:36:11.185 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49785.log.gz 2023-10-14T01:36:11.185 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2023-10-14T01:36:11.186 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48718.log.gz 2023-10-14T01:36:11.186 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53682.log 2023-10-14T01:36:11.186 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38546.log.gz 2023-10-14T01:36:11.187 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47422.log 2023-10-14T01:36:11.187 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2023-10-14T01:36:11.187 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53682.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-14T01:36:11.187 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53682.log.gz 2023-10-14T01:36:11.188 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53894.log 2023-10-14T01:36:11.188 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47422.log.gz 2023-10-14T01:36:11.188 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48628.log 2023-10-14T01:36:11.189 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50856.log 2023-10-14T01:36:11.189 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53894.log.gz 2023-10-14T01:36:11.189 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53210.log 2023-10-14T01:36:11.190 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48628.log.gz 2023-10-14T01:36:11.190 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53448.log 2023-10-14T01:36:11.190 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50856.log.gz 2023-10-14T01:36:11.190 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51964.log 2023-10-14T01:36:11.191 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53210.log.gz 2023-10-14T01:36:11.191 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45872.log 2023-10-14T01:36:11.191 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53448.log.gz 2023-10-14T01:36:11.191 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48235.log 2023-10-14T01:36:11.192 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51964.log.gz 2023-10-14T01:36:11.192 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54876.log 2023-10-14T01:36:11.192 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45872.log.gz 2023-10-14T01:36:11.193 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45319.log 2023-10-14T01:36:11.193 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48235.log.gz 2023-10-14T01:36:11.193 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54876.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30878.log 2023-10-14T01:36:11.194 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54876.log.gz 2023-10-14T01:36:11.194 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37656.log 2023-10-14T01:36:11.194 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45319.log.gz 2023-10-14T01:36:11.195 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.tmp-client.admin.30878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47134.log 2023-10-14T01:36:11.195 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30878.log.gz 2023-10-14T01:36:11.195 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47961.log 2023-10-14T01:36:11.196 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37656.log.gz 2023-10-14T01:36:11.196 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46150.log 2023-10-14T01:36:11.196 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47134.log.gz 2023-10-14T01:36:11.197 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50766.log 2023-10-14T01:36:11.197 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47961.log.gz 2023-10-14T01:36:11.197 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56180.log 2023-10-14T01:36:11.198 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46150.log.gz 2023-10-14T01:36:11.198 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39826.log 2023-10-14T01:36:11.198 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50766.log.gz 2023-10-14T01:36:11.199 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42009.log 2023-10-14T01:36:11.199 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56180.log.gz 2023-10-14T01:36:11.199 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38796.log 2023-10-14T01:36:11.200 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39826.log: /var/log/ceph/ceph-client.admin.42009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2023-10-14T01:36:11.200 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39826.log.gz 2023-10-14T01:36:11.200 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42009.log.gz 2023-10-14T01:36:11.201 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56154.log 2023-10-14T01:36:11.201 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38796.log.gz 2023-10-14T01:36:11.201 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54160.log 2023-10-14T01:36:11.202 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gz 2023-10-14T01:36:11.202 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56154.log.gz 2023-10-14T01:36:11.202 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45353.log 2023-10-14T01:36:11.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49209.log 2023-10-14T01:36:11.203 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54160.log.gz 2023-10-14T01:36:11.203 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48872.log 2023-10-14T01:36:11.204 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45353.log.gz 2023-10-14T01:36:11.204 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55837.log 2023-10-14T01:36:11.204 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49209.log.gz 2023-10-14T01:36:11.205 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48052.log 2023-10-14T01:36:11.205 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48872.log.gz 2023-10-14T01:36:11.205 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54481.log 2023-10-14T01:36:11.206 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55837.log.gz 2023-10-14T01:36:11.206 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2023-10-14T01:36:11.206 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48052.log.gz 2023-10-14T01:36:11.206 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53132.log 2023-10-14T01:36:11.207 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54481.log.gz 2023-10-14T01:36:11.207 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46468.log 2023-10-14T01:36:11.207 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2023-10-14T01:36:11.208 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53132.log.gz 2023-10-14T01:36:11.208 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50119.log 2023-10-14T01:36:11.208 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46957.log 2023-10-14T01:36:11.208 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46468.log.gz 2023-10-14T01:36:11.209 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.50119.log: /var/log/ceph/ceph-client.admin.52056.log 2023-10-14T01:36:11.209 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50119.log.gz 2023-10-14T01:36:11.209 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34613.log 2023-10-14T01:36:11.210 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46957.log.gz 2023-10-14T01:36:11.210 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52056.log.gz 2023-10-14T01:36:11.210 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42920.log 2023-10-14T01:36:11.210 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.34613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39536.log 2023-10-14T01:36:11.211 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34613.log.gz 2023-10-14T01:36:11.211 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53051.log 2023-10-14T01:36:11.211 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42920.log.gz 2023-10-14T01:36:11.211 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40045.log 2023-10-14T01:36:11.212 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39536.log.gz 2023-10-14T01:36:11.212 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48963.log 2023-10-14T01:36:11.212 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53051.log.gz 2023-10-14T01:36:11.213 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47287.log 2023-10-14T01:36:11.213 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40045.log.gz 2023-10-14T01:36:11.213 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46765.log 0.0% 2023-10-14T01:36:11.213 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48963.log.gz 2023-10-14T01:36:11.214 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52188.log 2023-10-14T01:36:11.214 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47287.log.gz 2023-10-14T01:36:11.214 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50702.log 2023-10-14T01:36:11.214 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46765.log.gz 2023-10-14T01:36:11.215 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55811.log 2023-10-14T01:36:11.215 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52188.log.gz 2023-10-14T01:36:11.215 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53946.log 2023-10-14T01:36:11.216 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50702.log.gz 2023-10-14T01:36:11.216 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46825.log 2023-10-14T01:36:11.216 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55811.log.gz 2023-10-14T01:36:11.217 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41083.log 2023-10-14T01:36:11.217 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53946.log.gz 2023-10-14T01:36:11.217 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45668.log 2023-10-14T01:36:11.218 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46825.log.gz 2023-10-14T01:36:11.218 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46580.log 2023-10-14T01:36:11.218 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41083.log.gz 2023-10-14T01:36:11.219 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45781.log 2023-10-14T01:36:11.219 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45668.log.gz 2023-10-14T01:36:11.219 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42489.log/var/log/ceph/ceph-client.admin.46580.log: 2023-10-14T01:36:11.220 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46580.log.gz 2023-10-14T01:36:11.220 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53367.log 2023-10-14T01:36:11.220 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45781.log.gz 2023-10-14T01:36:11.221 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44553.log 2023-10-14T01:36:11.221 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42489.log.gz 2023-10-14T01:36:11.221 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48841.log 2023-10-14T01:36:11.222 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53367.log.gz 2023-10-14T01:36:11.222 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55032.log 2023-10-14T01:36:11.222 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44553.log.gz 2023-10-14T01:36:11.223 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48447.log 2023-10-14T01:36:11.223 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48841.log.gz 2023-10-14T01:36:11.223 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44432.log 2023-10-14T01:36:11.224 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55032.log.gz 2023-10-14T01:36:11.224 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46885.log 2023-10-14T01:36:11.224 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48447.log.gz 2023-10-14T01:36:11.225 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47102.log 2023-10-14T01:36:11.225 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44432.log.gz 2023-10-14T01:36:11.225 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56263.log 2023-10-14T01:36:11.226 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46885.log.gz 2023-10-14T01:36:11.226 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43103.log 2023-10-14T01:36:11.227 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47102.log.gz 2023-10-14T01:36:11.227 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56263.log.gz 2023-10-14T01:36:11.227 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53604.log 2023-10-14T01:36:11.228 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49936.log 2023-10-14T01:36:11.228 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43103.log.gz 2023-10-14T01:36:11.228 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53604.log.gz 2023-10-14T01:36:11.229 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56023.log 2023-10-14T01:36:11.229 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45506.log 2023-10-14T01:36:11.229 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49936.log.gz 2023-10-14T01:36:11.230 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56023.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56023.log.gz 2023-10-14T01:36:11.230 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38779.log 2023-10-14T01:36:11.230 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55274.log 2023-10-14T01:36:11.231 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45506.log.gz 2023-10-14T01:36:11.231 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42830.log 2023-10-14T01:36:11.231 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38779.log.gz 2023-10-14T01:36:11.232 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44889.log 2023-10-14T01:36:11.232 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55274.log.gz 2023-10-14T01:36:11.232 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41275.log 2023-10-14T01:36:11.233 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42830.log.gz 2023-10-14T01:36:11.233 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47728.log 2023-10-14T01:36:11.233 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44889.log.gz 2023-10-14T01:36:11.234 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45136.log 2023-10-14T01:36:11.234 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41275.log.gz 2023-10-14T01:36:11.234 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52739.log 2023-10-14T01:36:11.234 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47728.log.gz 2023-10-14T01:36:11.235 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45136.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45136.log.gz 2023-10-14T01:36:11.235 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55866.log 2023-10-14T01:36:11.235 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55729.log 2023-10-14T01:36:11.235 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52739.log.gz 2023-10-14T01:36:11.236 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55866.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55866.log.gz 2023-10-14T01:36:11.236 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40394.log 2023-10-14T01:36:11.236 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55729.log.gz 2023-10-14T01:36:11.237 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2023-10-14T01:36:11.237 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40394.log.gz 2023-10-14T01:36:11.237 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43163.log 2023-10-14T01:36:11.237 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46334.log 2023-10-14T01:36:11.238 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42674.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2023-10-14T01:36:11.238 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37835.log 2023-10-14T01:36:11.238 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43163.log.gz 2023-10-14T01:36:11.238 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46714.log 2023-10-14T01:36:11.239 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46334.log.gz 2023-10-14T01:36:11.239 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46981.log 2023-10-14T01:36:11.239 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37835.log.gz 2023-10-14T01:36:11.240 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51218.log 2023-10-14T01:36:11.240 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46714.log.gz 2023-10-14T01:36:11.240 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48173.log 2023-10-14T01:36:11.240 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46981.log.gz 2023-10-14T01:36:11.241 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41638.log 2023-10-14T01:36:11.241 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51218.log.gz 2023-10-14T01:36:11.241 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43696.log 2023-10-14T01:36:11.241 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48173.log.gz 2023-10-14T01:36:11.242 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51038.log 2023-10-14T01:36:11.242 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41638.log.gz 2023-10-14T01:36:11.242 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45423.log 2023-10-14T01:36:11.242 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43696.log.gz 2023-10-14T01:36:11.243 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38469.log 2023-10-14T01:36:11.243 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51038.log.gz 2023-10-14T01:36:11.243 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56206.log 2023-10-14T01:36:11.244 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45423.log.gz 2023-10-14T01:36:11.244 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49420.log 2023-10-14T01:36:11.244 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38469.log.gz 2023-10-14T01:36:11.245 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54427.log 2023-10-14T01:36:11.245 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56206.log.gz 2023-10-14T01:36:11.245 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38654.log 2023-10-14T01:36:11.246 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49420.log.gz 2023-10-14T01:36:11.246 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49968.log 2023-10-14T01:36:11.246 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54427.log.gz 2023-10-14T01:36:11.247 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38654.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38654.log.gz 2023-10-14T01:36:11.247 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39302.log 2023-10-14T01:36:11.247 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46194.log 2023-10-14T01:36:11.248 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49968.log.gz 2023-10-14T01:36:11.248 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39302.log.gz 2023-10-14T01:36:11.248 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43740.log 2023-10-14T01:36:11.249 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46029.log 2023-10-14T01:36:11.249 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46194.log.gz 2023-10-14T01:36:11.249 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38671.log 2023-10-14T01:36:11.250 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43740.log.gz 2023-10-14T01:36:11.250 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43792.log 2023-10-14T01:36:11.250 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46029.log.gz 2023-10-14T01:36:11.250 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48205.log 2023-10-14T01:36:11.251 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38671.log.gz 2023-10-14T01:36:11.251 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55222.log 2023-10-14T01:36:11.251 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43792.log.gz 2023-10-14T01:36:11.251 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37359.log 2023-10-14T01:36:11.252 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48205.log.gz 2023-10-14T01:36:11.252 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55222.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55222.log.gz 2023-10-14T01:36:11.252 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51369.log 2023-10-14T01:36:11.253 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41423.log 2023-10-14T01:36:11.253 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37359.log.gz 2023-10-14T01:36:11.253 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54533.log 2023-10-14T01:36:11.254 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51369.log.gz 2023-10-14T01:36:11.254 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41423.log.gz 2023-10-14T01:36:11.254 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39740.log 2023-10-14T01:36:11.255 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55702.log 2023-10-14T01:36:11.255 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54533.log.gz 2023-10-14T01:36:11.255 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52921.log 2023-10-14T01:36:11.256 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39740.log.gz 2023-10-14T01:36:11.256 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43521.log 2023-10-14T01:36:11.256 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55702.log.gz 2023-10-14T01:36:11.257 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39336.log 2023-10-14T01:36:11.257 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52921.log.gz 2023-10-14T01:36:11.257 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50329.log 2023-10-14T01:36:11.258 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43521.log.gz 2023-10-14T01:36:11.258 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39194.log 2023-10-14T01:36:11.258 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39336.log.gz 2023-10-14T01:36:11.259 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37582.log 2023-10-14T01:36:11.259 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50329.log.gz 2023-10-14T01:36:11.259 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2023-10-14T01:36:11.260 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39194.log.gz 2023-10-14T01:36:11.260 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49815.log 2023-10-14T01:36:11.260 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37582.log.gz 2023-10-14T01:36:11.261 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49299.log 2023-10-14T01:36:11.261 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2023-10-14T01:36:11.261 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55300.log 2023-10-14T01:36:11.262 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49815.log.gz 2023-10-14T01:36:11.262 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52257.log 2023-10-14T01:36:11.262 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49299.log.gz 2023-10-14T01:36:11.262 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54401.log 2023-10-14T01:36:11.263 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55300.log.gz 2023-10-14T01:36:11.263 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40340.log 2023-10-14T01:36:11.263 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52257.log.gz 2023-10-14T01:36:11.264 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50088.log 2023-10-14T01:36:11.264 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54401.log.gz 2023-10-14T01:36:11.264 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49875.log 2023-10-14T01:36:11.265 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40340.log.gz 2023-10-14T01:36:11.265 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39228.log 2023-10-14T01:36:11.265 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50088.log.gz 2023-10-14T01:36:11.265 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2023-10-14T01:36:11.266 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49875.log.gz 2023-10-14T01:36:11.266 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55439.log 2023-10-14T01:36:11.266 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39228.log.gz 2023-10-14T01:36:11.267 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51553.log 2023-10-14T01:36:11.267 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2023-10-14T01:36:11.267 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40929.log 2023-10-14T01:36:11.267 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55439.log.gz 2023-10-14T01:36:11.268 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41251.log 2023-10-14T01:36:11.268 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51553.log.gz 2023-10-14T01:36:11.268 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50886.log 2023-10-14T01:36:11.268 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40929.log.gz 2023-10-14T01:36:11.269 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40515.log 2023-10-14T01:36:11.269 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41251.log.gz 2023-10-14T01:36:11.269 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41223.log 2023-10-14T01:36:11.269 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50886.log.gz 2023-10-14T01:36:11.270 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gzgzip 2023-10-14T01:36:11.270 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49269.log 2023-10-14T01:36:11.270 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52166.log 2023-10-14T01:36:11.271 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41223.log.gz 2023-10-14T01:36:11.271 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38600.log 2023-10-14T01:36:11.271 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.49269.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49269.log.gz 2023-10-14T01:36:11.271 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40539.log 2023-10-14T01:36:11.272 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52166.log.gz 2023-10-14T01:36:11.272 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52231.log 2023-10-14T01:36:11.272 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38600.log.gz 2023-10-14T01:36:11.272 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54719.log 2023-10-14T01:36:11.273 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40539.log.gz 2023-10-14T01:36:11.273 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39444.log 2023-10-14T01:36:11.273 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52231.log.gz 2023-10-14T01:36:11.274 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54719.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.54719.log.gz 2023-10-14T01:36:11.274 INFO:teuthology.orchestra.run.smithi183.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39390.log 2023-10-14T01:36:11.274 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44716.log 2023-10-14T01:36:11.274 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39444.log.gz 2023-10-14T01:36:11.275 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38580.log 2023-10-14T01:36:11.275 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39390.log.gz 2023-10-14T01:36:11.275 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52973.log 2023-10-14T01:36:11.276 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44716.log.gz 2023-10-14T01:36:11.276 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55169.log 2023-10-14T01:36:11.276 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38580.log.gz 2023-10-14T01:36:11.277 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53656.log 2023-10-14T01:36:11.277 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52973.log.gz 2023-10-14T01:36:11.278 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38489.log 2023-10-14T01:36:11.278 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55169.log.gz 2023-10-14T01:36:11.278 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51855.log 2023-10-14T01:36:11.279 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53656.log.gz 2023-10-14T01:36:11.279 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2023-10-14T01:36:11.279 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38489.log.gz 2023-10-14T01:36:11.280 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44011.log 2023-10-14T01:36:11.280 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51855.log.gz 2023-10-14T01:36:11.280 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2023-10-14T01:36:11.281 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gz 2023-10-14T01:36:11.281 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44011.log.gz 2023-10-14T01:36:11.281 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54296.log 2023-10-14T01:36:11.282 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41856.log 2023-10-14T01:36:11.282 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2023-10-14T01:36:11.282 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48780.log 2023-10-14T01:36:11.283 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54296.log.gz 2023-10-14T01:36:11.283 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.41856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40839.log 2023-10-14T01:36:11.283 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41856.log.gz 2023-10-14T01:36:11.284 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.48780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46396.log 2023-10-14T01:36:11.284 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48780.log.gz 2023-10-14T01:36:11.284 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.40839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52000.log 2023-10-14T01:36:11.285 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40839.log.gz 2023-10-14T01:36:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.46396.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46396.log.gz 2023-10-14T01:36:11.285 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44123.log 2023-10-14T01:36:11.286 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43073.log 2023-10-14T01:36:11.286 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52000.log.gz 2023-10-14T01:36:11.286 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54954.log 2023-10-14T01:36:11.287 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44123.log.gz 2023-10-14T01:36:11.287 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47178.log 2023-10-14T01:36:11.287 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43073.log.gz 2023-10-14T01:36:11.288 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42428.log 2023-10-14T01:36:11.288 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54954.log.gz 2023-10-14T01:36:11.288 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51735.log 2023-10-14T01:36:11.289 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47178.log.gz 2023-10-14T01:36:11.289 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52491.log 2023-10-14T01:36:11.289 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42428.log.gz 2023-10-14T01:36:11.290 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55971.log 2023-10-14T01:36:11.290 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51735.log.gz 2023-10-14T01:36:11.290 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52491.log.gz 2023-10-14T01:36:11.291 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53789.log 2023-10-14T01:36:11.291 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55971.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55971.log.gz 2023-10-14T01:36:11.291 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43663.log 2023-10-14T01:36:11.292 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50917.log 2023-10-14T01:36:11.292 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53789.log.gz 2023-10-14T01:36:11.292 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.43663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45902.log 2023-10-14T01:36:11.292 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43663.log.gz 2023-10-14T01:36:11.293 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52092.log 2023-10-14T01:36:11.293 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50917.log.gz 2023-10-14T01:36:11.293 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.45902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54349.log 2023-10-14T01:36:11.293 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45902.log.gz 2023-10-14T01:36:11.294 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2023-10-14T01:36:11.294 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52092.log.gz 2023-10-14T01:36:11.294 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.54349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53630.log 2023-10-14T01:36:11.294 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54349.log.gz 2023-10-14T01:36:11.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51982.log 2023-10-14T01:36:11.295 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2023-10-14T01:36:11.295 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47812.log 2023-10-14T01:36:11.296 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53630.log.gz 2023-10-14T01:36:11.296 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.51982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47478.log 2023-10-14T01:36:11.296 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51982.log.gz 2023-10-14T01:36:11.296 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56075.log 2023-10-14T01:36:11.297 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47812.log.gz 2023-10-14T01:36:11.297 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.47478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2023-10-14T01:36:11.297 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47478.log.gz 2023-10-14T01:36:11.297 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.56075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50732.log 2023-10-14T01:36:11.298 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56075.log.gz 2023-10-14T01:36:11.298 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.39481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42459.log 2023-10-14T01:36:11.298 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2023-10-14T01:36:11.298 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.50732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44859.log 2023-10-14T01:36:11.299 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50732.log.gz 2023-10-14T01:36:11.299 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55518.log 2023-10-14T01:36:11.299 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42459.log.gz 2023-10-14T01:36:11.300 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37549.log 2023-10-14T01:36:11.300 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44859.log.gz 2023-10-14T01:36:11.300 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.55518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53420.log 2023-10-14T01:36:11.300 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55518.log.gz 2023-10-14T01:36:11.301 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.37549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38617.log 2023-10-14T01:36:11.301 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37549.log.gz 2023-10-14T01:36:11.301 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44367.log 2023-10-14T01:36:11.302 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53420.log.gz 2023-10-14T01:36:11.302 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.38617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53920.log 2023-10-14T01:36:11.302 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38617.log.gz 2023-10-14T01:36:11.303 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.44367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52869.log 2023-10-14T01:36:11.303 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44367.log.gz 2023-10-14T01:36:11.303 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.53920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42223.log 2023-10-14T01:36:11.304 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53920.log.gz 2023-10-14T01:36:11.304 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.52869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34554.log 2023-10-14T01:36:11.304 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52869.log.gz 2023-10-14T01:36:11.305 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.42223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42223.log.gz 2023-10-14T01:36:11.305 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.34554.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34554.log.gz 2023-10-14T01:36:11.306 INFO:teuthology.orchestra.run.smithi183.stderr: 90.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-14T01:36:11.548 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-10-14T01:36:22.885 INFO:teuthology.orchestra.run.smithi183.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-14T01:36:34.310 INFO:teuthology.orchestra.run.smithi183.stderr: 90.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-14T02:18:24.110 INFO:teuthology.orchestra.run.smithi183.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-14T02:22:34.675 INFO:teuthology.orchestra.run.smithi183.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-14T02:29:56.991 INFO:teuthology.orchestra.run.smithi183.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-14T02:29:57.021 INFO:teuthology.orchestra.run.smithi183.stderr: 2023-10-14T02:29:57.021 INFO:teuthology.orchestra.run.smithi183.stderr:real 53m46.375s 2023-10-14T02:29:57.021 INFO:teuthology.orchestra.run.smithi183.stderr:user 44m4.864s 2023-10-14T02:29:57.022 INFO:teuthology.orchestra.run.smithi183.stderr:sys 2m13.954s 2023-10-14T02:29:57.022 INFO:tasks.ceph:Archiving logs... 2023-10-14T02:29:57.024 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259/remote/smithi183/log 2023-10-14T02:29:57.025 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-14T02:32:23.824 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-14T02:32:23.838 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-14T02:32:23.839 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-14T02:32:23.894 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-14T02:32:23.906 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-14T02:32:23.907 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-14T02:32:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: remote refid st t when poll reach delay offset jitter 2023-10-14T02:32:23.969 INFO:teuthology.orchestra.run.smithi183.stdout:============================================================================== 2023-10-14T02:32:23.969 INFO:teuthology.orchestra.run.smithi183.stdout:*hv01.front.sepi 44.4.53.2 2 u 86 256 377 0.154 -1.382 0.385 2023-10-14T02:32:23.969 INFO:teuthology.orchestra.run.smithi183.stdout:+hv02.front.sepi 63.231.80.2 3 u 129 256 377 0.142 -1.999 0.160 2023-10-14T02:32:23.970 INFO:teuthology.orchestra.run.smithi183.stdout:+hv03.front.sepi 74.6.168.72 3 u 100 256 377 0.191 -1.548 0.224 2023-10-14T02:32:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-10-14T02:32:23.972 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-14T02:32:23.981 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-14T02:32:23.981 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-14T02:32:24.008 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-14T02:32:24.041 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-14T02:32:24.073 INFO:teuthology.task.internal:Duration was 9758.883320 seconds 2023-10-14T02:32:24.074 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-14T02:32:24.109 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-14T02:32:24.109 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-14T02:32:24.410 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-14T02:32:24.411 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2023-10-14T02:32:24.411 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-14T02:32:24.495 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-14T02:32:24.496 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-14T02:32:24.595 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-14T02:32:24.595 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-14T02:32:25.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-14T02:32:25.130 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-14T02:32:25.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-14T02:32:25.182 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-14T02:32:25.197 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-14T02:32:25.224 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2023-10-14T02:32:25.248 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-14T02:32:25.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T02:32:25.297 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-14T02:32:25.308 INFO:teuthology.task.internal:Transferring archived files... 2023-10-14T02:32:25.310 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259/remote/smithi183 2023-10-14T02:32:25.310 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-14T02:32:25.371 INFO:teuthology.task.internal:Removing archive directory... 2023-10-14T02:32:25.372 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-14T02:32:25.417 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-14T02:32:25.449 INFO:teuthology.task.internal:Not uploading archives. 2023-10-14T02:32:25.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-14T02:32:25.486 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-14T02:32:25.486 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-14T02:32:25.533 INFO:teuthology.orchestra.run.smithi183.stdout: 655472 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Oct 14 02:32 /home/ubuntu/cephtest 2023-10-14T02:32:25.533 INFO:teuthology.orchestra.run.smithi183.stdout: 791994 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Oct 13 23:59 /home/ubuntu/cephtest/mnt.0 2023-10-14T02:32:25.534 INFO:teuthology.orchestra.run.smithi183.stdout: 792380 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Oct 14 00:00 /home/ubuntu/cephtest/mnt.0/client.0 2023-10-14T02:32:25.534 INFO:teuthology.orchestra.run.smithi183.stdout: 792425 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Oct 14 00:00 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-10-14T02:32:25.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-14T02:32:25.537 INFO:teuthology.orchestra.run.smithi183.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-10-14T02:32:25.538 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi183 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-14T02:32:25.545 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-14T02:32:25.557 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-14T02:32:25.616 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi183 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-10-14T02:32:25.616 INFO:teuthology.nuke:Checking targets against current locks 2023-10-14T02:32:25.667 DEBUG:teuthology.nuke:shortname: smithi183 2023-10-14T02:32:25.668 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-14T02:32:25.694 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dis-2023-10-13_23:35:38-rbd-wip-dis-testing-distro-default-smithi/7427259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-13 23:41:15.894935', 'locked_by': 'scheduled_dis@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH/vh7O2KCB/0ptwMSzzgY+G3dboygsXf4i9tzvtMR/JZ7YzaIu2HcgDfLzHJ7Tqa+552PeXue3WWel93DCIE20='} 2023-10-14T02:32:25.758 INFO:teuthology.orchestra.console.smithi183:Power off 2023-10-14T02:32:25.759 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-14T02:32:25.799 DEBUG:teuthology.orchestra.console.smithi183:power off output: Chassis Power Control: Down/Off 2023-10-14T02:32:25.800 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T02:32:25.829 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-10-14T02:32:29.831 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T02:32:29.942 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is on 2023-10-14T02:32:33.945 DEBUG:teuthology.orchestra.console.smithi183:pexpect command: ipmitool -H smithi183.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-14T02:32:34.055 DEBUG:teuthology.orchestra.console.smithi183:check power output: Chassis Power is off 2023-10-14T02:32:34.156 INFO:teuthology.orchestra.console.smithi183:Power off completed 2023-10-14T02:32:34.262 INFO:teuthology.run:Summary data: description: rbd/cli/{base/install clusters/{fixed-1 openstack} conf/{disable-pool-app} data-pool/none features/journaling msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{ubuntu_latest} workloads/rbd_support_module_recovery} duration: 9758.883319616318 failure_reason: 'Command failed (workunit test rbd/rbd_support_module_recovery.sh) on smithi183 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=735f68d7bf91d643813a8420fc2b11441f4bd8c4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/rbd_support_module_recovery.sh''' flavor: default owner: scheduled_dis@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=146b1c4b7a81412e8ebb7743359e5ed9 status: fail success: false 2023-10-14T02:32:34.263 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-14T02:32:34.332 INFO:teuthology.run:FAIL